Hearing aid system for estimating acoustic transfer functions

ABSTRACT

A hearing aid system comprises a hearing aid, and a portable auxiliary device&#39; adapted to establish a communication link between them. The hearing aid comprises a microphone providing an electric input signal, a signal processor, and an output unit. The auxiliary device comprises a microphone providing an auxiliary electric input signal, and a user control interface allowing a user to initiate a specific calibration mode of operation of the hearing aid system. The signal processor of the hearing aid is configured to receive corresponding time segments of said electric input signal and said auxiliary electric input signal to provide an estimate of an acoustic transfer function from said microphone of said auxiliary device to said microphone of said hearing aid. A method of operating a hearing aid system is further disclosed. The invention may e.g. be used in various applications related to own voice detection and estimation.

SUMMARY

The present application deals with hearing aid systems and hearing aidsor headsets. The disclosure is mainly related to functionality centredaround own-voice detection and/or own voice estimation, i.e., e.g.focused on detecting when a hearing aid (HA) user speaks, and/orown-voice retrieval, i.e., e.g. retrieving an enhanced version of theHA-user's voice signal when spoken in a potentially acoustically noisysituation. Own-voice detection and retrieval are key technologies, e.g.for hands-free telephony, voice-controlled hearing devices, etc. Theideas of the present disclosure focus on personalized solutions, i.e.,methods for improving performance of different algorithms, by tailoringthem to particular (acoustic) characteristics of the specific user,e.g., voice characteristics, head and torso characteristics, etc.

To do so, it is proposed to use a mobile phone (or any other mobile(e.g. portable, e.g. wearable) communication device comprising amicrophone and/or a loudspeaker) as

a) a wireless microphone, and/or

b) a wireless loudspeaker.

The underlying assumption is that most hearing aid (HA) or headset (HS)users are (or will be) equipped with a mobile phone (or similar portable(e.g. wearable) device) and will be quite proficient in using it.

Using a mobile phone (or similar mobile device) as a wireless microphoneallows a user

i) to pick up a voice sample of the user (in the mobile device, and/orin the hearing aid).

ii) to estimate the acoustic transfer function from (just outside) themouth of the HA- or HS-user to each of the microphones in the HAs (or ofthe HS) (we denote these acoustic transfer functions as Own-VoiceTransfer Functions (OVTFs)),

Using a mobile phone (or similar mobile device) as a wirelessloudspeaker allows a user

iii) to estimate the acoustic transfer function from an arms distance,e.g. frontal to the HA (or HS) user, e.g. a typical position of aconversation partner, to the microphones of the HAs (or the HS) (frontalHead Related Transfer Function (HRTF)).

EP2928215A1 describes the use of a mobile device comprising aloudspeaker to provide personalized beamforming, see e.g. FIG. 1 ofEP2928215A1 and the corresponding description.

US20120063610A1 deals with wireless audio streaming scenarios whereinthe acoustical audio signal is present in parallel to a correspondingwireless electromagnetic signal, e.g. audio streaming from a TV, audiostreaming in a classroom, etc. The two representations are used incombination to present to the user in various examples, one being usedto improve the other. In one example, the acoustically propagated signalis improved by using the ‘clean’ wirelessly received signal for noisereduction before being presented to the user. US20120063610A1 indicatesthat a difference in transfer function between an acoustic and anelectromagnetic propagation path of sound from another device to ahearing aid microphone can be estimated.

US20190115041A1 deals with the same scenario as US20120063610A1(reception of respective acoustically and wirelessly propagatedrepresentations of a target signal). A delay between a wirelessly and anacoustically received target signal is estimated. This may represent anestimate of a transfer function from the microphone of the auxiliarydevice to a microphone of the hearing aid.

In the following, various ideas and applications are presented in thecontext of a hearing aid system. However, they are equally applicable toother head-worn communication devices, e.g., headsets, helmets, etc.

A First Hearing Aid System

In an aspect, a hearing aid system comprising a hearing aid AND anauxiliary device is provided.

In an aspect of the present application, a hearing aid system isprovided. The hearing aid system may comprise A) a (e.g. at least one)hearing aid adapted for being worn by a user at an ear, or in an ear, ofthe user, or for being fully or partially implanted in the head at anear of the user, and B) a portable auxiliary device, e.g. acommunication device, such as a smartphone. The hearing aid system maybe adapted to establish a communication link between the hearing aid andthe auxiliary device to provide that data, e.g. control signals, statussignals, and/or audio signals, can be exchanged between them orforwarded from one to the other. The hearing aid may comprise at leastone microphone for picking up sound from the environment of the hearingaid and configured to provide corresponding at least one electric inputsignal representative of the sound. The hearing aid may further comprisea signal processor configured to process the electric input signal or asignal derived therefrom and to provide a processed signal. The hearingaid may further comprise an output unit, e.g. comprising a loudspeaker,for presenting stimuli perceivable as sound to the user representativeof the processed signal. The auxiliary device may comprise at least onemicrophone for picking up sound from the environment of the auxiliarydevice and to provide corresponding at least one auxiliary electricinput signal representative of the sound. The auxiliary device mayfurther comprise a user control interface allowing a user to initiate aspecific calibration mode of operation of the hearing aid system. Thesignal processor of the hearing aid may be configured to comparecorresponding current time segments of the at least one electric inputsignal, and the at least one auxiliary electric input signal, orcorresponding transforms (or selected frequency ranges) thereof, and toprovide an estimate of a transfer function from the at least onemicrophone of the auxiliary device to the at least one microphone (e.g.two or more) of the hearing aid.

Thereby an improved hearing aid system may be provided.

The term ‘corresponding transforms’ may e.g. include Fourier transforms,e.g. Short-time Fourier transform (STFT), Discrete-time Fouriertransform (DTFT), etc., or other transforms, such as e.g. Laplacetransform, cosine or sine transform (e.g. Discrete cosine or sinetransform), etc.

When transmitting electric signals representing audio between devices,e.g. from the ‘at least one auxiliary electric signal’ from theauxiliary device to the hearing aid (or vice versa), it may beconsidered (in an attempt to limit transmission bandwidth, and thuspower) to only transmit selected frequency ranges (e.g. the mostimportant frequency ranges, e.g. frequency ranges containing (e.g.important) speech components).

When the at least one microphone of the auxiliary device is positionedin proximity of, e.g. in front of, the user's mouth, the transferfunction may represent an own voice transfer function (OVTF), when theuser—during the own voice calibration mode of operation of the hearingaid system—raises his or her voice, e.g. speaks. Preferably, themicrophone of the auxiliary device is positioned less than a maximumdistance from the mouth of the user when the user speaks. The maximumdistance may e.g. be 0.1 m, such as 0.05 m, or 0.02 m. The userinterface may be configured to initiate a measurement of respective ownvoice transfer function(s) in the specific own voice calibration mode ofoperation of the hearing aid system. The hearing aid system may e.g. beconfigured to instruct the user—e.g. via the user interface of theauxiliary device—how to position (and/or orientate) the auxiliary devicerelative to the user, and/or to speak a certain phrase or sentence (e.g.with a particular vocal effort, e.g. loud or soft, etc., e.g. independence of a current noise level around the user), cf. e.g. FIG. 4B.

The hearing aid system may comprise at least one voice activity detectorallowing to classify an electric input signal representing sound (e.g.time segments thereof) as originating from a human voice or notoriginating from a human voice, possibly to classify it as own voice ornot own voice, e.g. classify the sound as speech or not speech. Theclassification of time segments may be performed on a frequency sub-bandlevel. The classification may be binary (e.g. 0 or 1, or ‘no’ or ‘yes’,etc.) or probabilistic (e.g. a value between 0 and 1). Own voice is inthe present context taken to mean the voice of the wearer of the hearingaid (‘the user’).

The initiation of the specific own voice calibration mode of operationof the hearing aid system may e.g. include a simultaneous start of arecording of the user's own voice (e.g. in the form of time segments ofelectric signals from the available microphones), e.g. when an own voicedetector indicates that the user' own voice is present, or is presentwith a probability above a, e.g. predetermined (or adaptivelydetermined), threshold value. The user interface is configured to allowa user to initiate a specific calibration, e.g. to start recording (e.g.storing) corresponding current time segments of the at least oneelectric input signal, and the at least one auxiliary electric inputsignal.

The auxiliary device may be configured to generate a calibration controlsignal upon initiation of a specific calibration from the userinterface. The auxiliary device may be configured to transmit thecalibration control signal to the hearing aid. The auxiliary device maybe configured to start and/or stop recording a time segment of the atleast one auxiliary electric input signal in dependence of thecalibration control signal. The hearing aid may be configured to receivethe calibration control signal from the auxiliary device. The hearingaid may be configured to start and/or stop recording a time segment ofthe at least one electric input signal in dependence of the calibrationcontrol signal. The auxiliary device may be configured to start and/orstop transmitting a current time segment of the at least one auxiliaryelectric input signal to the hearing aid in dependence of thecalibration control signal.

The hearing aid may be configured to determine a (e.g. frequencydependent) transfer function based on the recorded time segments of theat least one electric input signal and the at least one auxiliaryelectric input signal. The hearing aid may be configured to determine atransfer function in dependence of the calibration control signal. Thehearing aid may be configured to determine a transfer function independence of an own voice control signal, e.g. in that only parts ofthe recorded time segments coinciding with an indication by the ownvoice control signal that the user's voice is present are used. Thehearing aid may be configured to determine a transfer function independence of the calibration control signal and the own voice controlsignal.

The hearing aid and the auxiliary device may comprise respective antennaand transceiver circuitry allowing the communication link between thehearing aid and the auxiliary device to be established.

The user interface may be configured to control functionality of thehearing aid system (including an initiation (and/or a termination) ofthe own voice calibration mode).

The corresponding (e.g. current) time segments of the at least oneelectric input signal, and the at least one auxiliary electric inputsignal, or corresponding transforms, or selected frequency rangesthereof, may be stored in a memory of the hearing aid system. The memorymay be distributed between the hearing aid and the auxiliary device (orlocated in another device or system). The current time segment of the atleast one electric input signal, or a transform thereof, may be storedin a memory of the hearing aid. The current time segment of the at leastone auxiliary electric input signal, or a transform thereof, may bestored in a memory of the auxiliary device. The current time segment ofthe at least one auxiliary electric input signal, or a transformthereof, may be transmitted to the hearing aid via the communicationlink. The current time segment of the at least one auxiliary electricinput signal may, when received in the hearing aid, be stored in thememory of the hearing aid. Based thereon, the two corresponding currenttime segments of the respective electric input signals may be used (e.g.on a frequency by frequency basis) to determine a (frequency dependent,acoustic) transfer function from the (at least one microphone of the)auxiliary device to an appropriate one of the at least one microphonesof the hearing aid. The thus determined (acoustic) transfer function(s)or a representation thereof may be stored in a memory or the hearing aidsystem, e.g. in the memory of the hearing aid.

The hearing aid may comprise at least a part of the memory wherein timesegments of the at least one electric input signal, and/or the at leastone auxiliary electric input signal, or corresponding transforms, orselected frequency ranges thereof, can be stored.

The hearing aid may comprise an own voice detector. The hearing aid maybe configured to only store the time segment of the at least oneelectric signal when the own voice detector indicates that the user'svoice is present (or present with a probability above a threshold value,e.g. 50%).

The auxiliary device may be configured to generate a calibration controlsignal upon initiation of the specific calibration from the userinterface.

The auxiliary device may be configured to transmit a current timesegment of the at least one auxiliary electric input signal to thehearing aid in dependence of the calibration control signal.

One of the at least one microphones of the hearing aid may be defined asa reference microphone. The hearing aid may comprise at least twomicrophones, each for picking up sound from the environment of thehearing aid and to provide corresponding at least two electric inputsignals representative of said sound. One of said at least twomicrophones may be defined as a reference microphone. The definition ofa reference microphone may be used to determine relative (acoustic)transfer functions for a given sound source location from the referencemicrophone to other microphones of the hearing aid system.

The auxiliary device may comprise a loudspeaker and the auxiliary devicemay be configured—in a separate far-field calibration mode ofoperation—to play a test sound signal to the environment of theauxiliary device and to transmit an electric version of the test soundto the hearing aid in dependence of an input from the user controlinterface. The user interface may be configured to initiate ameasurement of respective head-related transfer function(s) in thespecific far-field calibration mode of operation. The hearing aid systemmay be configured to instruct the user—e.g. via the user interface ofthe auxiliary device—how to act during the far-field calibration mode(e.g. how to position (and/or orientate) the auxiliary device relativeto the user), cf. e.g. FIG. 4C. In the far-field calibration mode, theauxiliary device is positioned at a preferred location relative to theuser (specifically to the hearing aid microphone(s) for which an(acoustic) transfer function is to be estimated), e.g. held in a hand,or located at or on a table or other support. The preferred location(e.g. distance, angle, etc.) relative to the user may be known inadvance, or estimated during calibration, e.g. using one or moresensors, e.g. of the auxiliary device and/or the hearing aid. Thehearing aid system may be configured to make data representative of theestimated location available (e.g. transmitted) to the hearing aid.Specifically, a distance between the auxiliary device and the hearingaid (e.g. between the loudspeaker of the auxiliary device and one of themicrophones, e.g. a reference microphone, of the hearing aid), may beestimated and stored, e.g. in the hearing aid. The distance may e.g. beestimated in the auxiliary device, and e.g. transmitted to the hearingdevice.

The hearing aid system may comprise a distance sensor for estimating adistance between the auxiliary device and the hearing aid. The auxiliarydevice may, in a calibration mode of operation where head-relatedtransfer functions of the user are estimated (the far-field calibrationmode), be configured to estimate a distance between the auxiliary device(e.g. the loudspeaker of the auxiliary device) and the hearing aid. Thedistance sensor may comprise an image sensor, e.g. a camera (such as ahigh-speed camera).

The hearing aid system may be configured to—in said specific far-fieldcalibration mode of operation—provide a test signal (e.g. by a signalgenerator, or by a stored waveform), which when played by saidloudspeaker provides said test sound signal.

The hearing aid system, e.g. the auxiliary device, may comprise a testsignal generator for providing a test signal, which when played by saidloudspeaker provides the test sound signal. The test signal generatormay be connectable to the loudspeaker and may e.g. be connected to theloudspeaker as part of the specific far-field calibration mode. The testsignal may be configured to comprise (or mimic) speech (to therebyensure that the relevant frequencies of speech are included). Thehearing aid may comprise a memory accessible to the signal processor,e.g. for—during the far-field mode of operation—storing a time segmentof the at least one electric signal and possibly of the at least oneauxiliary electric signal, and possibly of a representation of aresulting acoustic transfer function. A memory of the hearing aid system(e.g. the hearing aid or the auxiliary device or another device) maycomprise a test signal segment which when played by the loudspeaker ofthe auxiliary device provides the test sound signal.

The test sound signal may be configured to contain frequencies ofimportance for the application in question, e.g. in that it containsenergy at frequencies of importance, e.g. at speech frequencies, e.g.between 1 kHz and 5 kHz. It may be beneficial that the test sound signalis a chirp signal (e.g. a tonal signal whose frequency increases withtime). Thereby, the HRTF may be estimated using a procedure as e.g.outlined in [Farina, 2000]. The test sound signal may be adaptivelyconfigured in level relative to a current ambient noise level (to ensurea test sound signal to noise ratio above a certain (e.g. predefined)minimum SNR-threshold).

The hearing aid may be configured to—in said specific far-fieldcalibration mode of operation—store a time segment of said at least oneelectric signal when said test sound signal is picked up by said atleast one microphone of the hearing aid. The hearing aid system may beconfigured to—in said specific far-field calibration mode ofoperation—transmit said test sound signal to said hearing aid (e.g. viathe communication link). The hearing aid may be configured to receivethe test sound signal and to store it in the memory. The signalprocessor of the hearing aid may be configured to receive the test soundsignal and a time segment of the at least one electric signal (e.g. thetime segment received by a reference microphone) and (based thereon) todetermine a head related transfer function (HRTF) from the location ofthe loudspeaker to the at least one microphone (e.g. the referencemicrophone).

A hearing aid system wherein the signal processor of the hearing aid—insaid specific far-field calibration mode of operation—is configured toreceive said electric version of the test sound and said time segment ofsaid at least one electric signal and based thereon to determine anacoustic transfer function from the position of the loudspeaker of theauxiliary device to the at least one microphone of the hearing aid. Theacoustic transfer function from the position of the loudspeaker of theauxiliary device to the at least one microphone of the hearing aid ishere also termed the head related transfer function (for the acousticchannel from the location of the loudspeaker to the location of themicrophone in question).

The hearing aid system may comprise a carrier, e.g. a ‘selfie-stick’,adapted to receive and carry the auxiliary device, so that the auxiliarydevice can be positioned farther away from the user than an arm'slength. The auxiliary device may be attached to the carrier, e.g. theselfie stick. Based on correlation (e.g. determined by the hearing aid),e.g., between the electric signal(s) of the at least one hearing aidmicrophones and the electric signal(s) of the at least one microphone ofthe auxiliary device, the length of the selfie stick may be adjustedsuch that a desired distance between the hearing instrument microphonesand the phone in front of the user is obtained. Oppositely, a givendistance may be indicated by the correlation measurement. The hearingaid system may be configured to initiate the calibration measurementwhen a certain (e.g. predefined) distance (e.g. indicated by the uservia the user interface) is obtained. Hereby the user does not have toactively initiate the measurement. The user may be notified prior to thebeginning of the calibration measurement (to achieve that the user isnot moving during the measurement). Notification may happen via thephone screen, by audio from the phone, or via audio played via theoutput unit of the hearing aid.

The hearing aid may comprise a beamformer filter configured to provideone or more beamformers, where filter weights of said one or morebeamformers are personalized using said transfer functions. The one ormore beamformers may comprise an own voice beamformer aimed at pickingup the voice of the user (in which case the calibrated own voicetransfer functions are used in the determination of the filter weights).The one or more beamformers may comprise far-field-beamformer aimed atpicking up the voice of a communication partner (in which case thecalibrated head related transfer functions are used in the determinationof the filter weights).

The one or more beamformers may comprise an own voice beamformercomprising personalized filter weights, the own voice beamformer beingconfigured to enhance signals originating from the direction of theuser's mouth and to suppress sound signals from other directions. Theown voice beamformer may be configured to provide an estimate of theuser's own voice. The hearing aid may—e.g. in a telephone mode ofoperation—be configured to transmit the estimate of the user's own voicesignal to another device, e.g. to the auxiliary device (e.g. asmartphone).

The one or more beamformers may further comprise a beamformer comprisingpersonalized filter weights, the beamformer being configured to suppresssound signals from a far-field speaker.

The hearing aid may be constituted by or comprise a hearing instrumentfor compensating the user's hearing impairment.

The auxiliary device may be constituted by or comprise a remote control,a smartphone, or other portable or wearable electronic device, such as asmartwatch or the like.

The auxiliary device may be constituted by or comprise a remote controlfor controlling functionality and operation of the hearing aid system orhearing aid(s). The function of a remote control for the hearing aidsystem may be implemented in a smartphone. The auxiliary device, e.g.the smartphone, may be configured to run an application (APP) allowingto control functionality of the hearing aid via the auxiliary device.The hearing aid(s) may comprise an appropriate wireless interface to theauxiliary device, e.g. a smartphone, e.g. based on Bluetooth or someother standardized or proprietary scheme.

The auxiliary device may be constituted by or comprise an audio gatewaydevice adapted for receiving a multitude of audio signals (e.g. from anentertainment device, e.g. a TV or a music player, a telephoneapparatus, e.g. a mobile telephone or a computer, e.g. a PC) and adaptedfor allowing a user to select and/or combine an appropriate one of thereceived audio signals (or combination of signals) for transmission tothe hearing aid.

The hearing aid system may comprise two hearing aids adapted toimplement a binaural hearing aid system. The two hearing aids maycomprise appropriate antenna and transceiver circuitry allowing them toexchange data (e.g. audio and/or control data) between them. Thereby(expected) symmetries may be exploited (in particular in thedetermination of own voice transfer functions and far-field (headrelated) transfer functions from a front direction relative to theuser).

A Second Hearing Aid System

In a further aspect of the present application, a hearing aid system isprovided. The hearing aid system may comprise A) a (e.g. at least one)hearing aid adapted for being worn by a user, and B) a portableauxiliary device, e.g. a communication device, such as a smartphone. Thehearing aid system may be adapted to establish a communication linkbetween the hearing aid and the auxiliary device to provide that datacan be exchanged between them. The hearing aid may comprise at least onemicrophone for picking up sound from the environment of the hearing aidand configured to provide corresponding at least one electric inputsignal representative of the sound. The auxiliary device may comprise atleast one microphone for picking up sound from the environment of theauxiliary device and to provide corresponding at least one auxiliaryelectric input signal representative of the sound. The hearing aidsystem may further comprise a signal processor configured to—in aspecific own voice calibration mode of operation of the hearing aidsystem—compare corresponding current time segments of the at least oneelectric input signal, and the at least one auxiliary electric inputsignal, or corresponding transforms (or selected frequency ranges)thereof, wherein said corresponding current time segments are recordedwhile the user speaks, and to provide an estimate of a personalized ownvoice transfer function from the at least one microphone of theauxiliary device to the at least one microphone (e.g. two or more) ofthe hearing aid, when worn by the user. The at least one microphone ofthe auxiliary device is preferably positioned close to (e.g. less than0.1 m from) the user's mouth, when the user speaks in the own voicecalibration mode.

Thereby, a hearing aid system with improved functionality may beprovided.

The signal processor may be located in the hearing aid. The hearing aidmay be configured to receive the at least one auxiliary electric inputsignal from the auxiliary device. The signal processor of the hearingaid may be configured to receive corresponding time segments of the atleast one electric input signal and the at least one auxiliary electricinput signal, or corresponding transforms, or selected frequency regionsthereof, and to provide an estimate of a personalized own voice transferfunction from the at least one microphone of the auxiliary device to theat least one microphone of the hearing aid.

The signal processor may be located in the auxiliary device. Theauxiliary device may be configured to receive the at least one electricinput signal from the hearing aid. The signal processor of the auxiliarydevice may be configured to receive corresponding time segments of theat least one electric input signal and the at least one auxiliaryelectric input signal, or corresponding transforms, or selectedfrequency regions thereof, and to provide an estimate of a personalizedown voice transfer function from the at least one microphone of theauxiliary device to the at least one microphone of the hearing aid. Theauxiliary device may be configured to transmit the personalized ownvoice transfer function to the hearing aid (or to an external processingdevice).

The signal processor may be located in an external processing device(different from the hearing device and the auxiliary device). Theexternal processing device may be configured to receive the at least oneelectric input signal from the hearing aid as well as the at least oneauxiliary electric input signal from the auxiliary device. The signalprocessor of the external processing device may be configured to receivecorresponding time segments of the at least one electric input signaland the at least one auxiliary electric input signal, or correspondingtransforms, or selected frequency regions thereof, and to provide anestimate of a personalized own voice transfer function from the at leastone microphone of the auxiliary device to the at least one microphone ofthe hearing aid. The external processing device may be configured totransmit the personalized own voice transfer function to the hearingaid. The external processing device may be or form part of a stationarydevice, e.g. a charging station or a TV-adapter or similar accessorydevice for the hearing aid. The external processing device may beconfigured to have more processing capacity and more energy than thehearing aid.

The hearing aid may further comprise a hearing aid signal processorconfigured to process the electric input signal or a signal derivedtherefrom and to provide a processed signal. The hearing aid signalprocessor may include the signal processor of the hearing aid system.

The hearing aid may further comprise an output unit, e.g. comprising aloudspeaker, for presenting stimuli perceivable as sound to the userrepresentative of the processed signal.

The auxiliary device (and/or the external processing device) maycomprise a user control interface allowing a user to initiate thespecific own voice calibration mode of operation of the hearing aidsystem.

The hearing aid system may comprise one or more detectors configured todecide whether or not (or with what probability) the user is currentlywearing the hearing aid (or hearing aids of a binaural hearing aidsystem) and to provide a mode control signal indicative thereof. Thehearing aid system may be configured to only allow to enter the ownvoice calibration mode, in dependence of the mode control signal.

The hearing aid may comprise a beamformer filter configured to providean own voice beamformer comprising personalized filter weightsdetermined in dependence of said estimate of a personalized own voicetransfer function. The own voice beamformer may be configured to enhancesignals originating from the direction of the user's mouth relative tosound signals from other directions.

The features of the first hearing aid system described above in thedetailed description of embodiments and in the claims are intended to becombinable with the second hearing aid system as described above.

A Hearing Aid Configured to be Used in a Hearing Aid System

In a further aspect, a hearing aid configured to be used in the firstand second hearing aid system as described above, in the detaileddescription of embodiments and in the clams is provided by the presentdisclosure.

The hearing aid is adapted for being worn by a user at a or in an ear ofthe user, or for being fully or partially implanted in the head at anear of the user. The hearing aid is adapted to establish a communicationlink to an auxiliary device (e.g. a smartphone) to provide that data canbe exchanged between them or forwarded from one to the other. Thehearing aid may further comprise an input unit comprising at least onemicrophone for picking up sound from the environment of the hearing aidand to provide corresponding at least one electric input signalrepresentative of said sound, a signal processor configured to processsaid at least one electric input signal or a signal or signals derivedtherefrom and to provide a processed signal, and an output unit, e.g.comprising a loudspeaker, for presenting stimuli perceivable as sound tothe user representative of the processed signal.

The hearing aid may comprise antenna and transceiver circuitry allowingthe hearing aid to establish a communication link to an auxiliary deviceto provide that data can be exchanged between them or forwarded from oneto the other.

The hearing aid may comprise an output transducer for presenting stimuliperceivable as sound to the user representative of the processed signal.

The hearing aid may be configured to receive an auxiliary electric inputsignal provided by a microphone of said auxiliary device via saidcommunication link. The signal processor may—in a specific own voicecalibration mode of operation of the hearing aid system—be configured A)to receive corresponding time segments of the at least one electricinput signal, and the at least one auxiliary electric input signal, orcorresponding transforms, or selected frequency ranges thereof, and B)to provide an estimate of a personalized own voice transfer functionfrom said microphone of said auxiliary device to said at least onemicrophone of the hearing aid.

The signal processor may be configured to receive corresponding timesegments of the at least one electric input signal, and at least oneauxiliary electric input signal provided by a microphone of saidauxiliary device, or corresponding transforms, or selected frequencyranges thereof, and to provide an estimate of a transfer function fromsaid at least one microphone of said auxiliary device to said at leastone microphone of the hearing aid.

Features of the hearing aid system as described above, in the detaileddescription of embodiments and in the clams may be combined with thehearing aid (where appropriate).

The hearing aid may comprise a beamformer filter configured to providean own voice beamformer comprising personalized filter weightsdetermined in dependence of said estimate of a personalized own voicetransfer function. The own voice beamformer may be used in variousapplications where a good estimate of the speech of the user is needed,e.g. handsfree telephony, speech recognition (wake-word-, keyworddetection), etc.

The hearing aid may comprise a beamformer filter configured to providean own voice-cancelling beamformer comprising personalized filterweights determined in dependence of said estimate of a personalized ownvoice transfer function. The own voice-cancelling beamformer may beuseful in situations where only sounds from the environment are ofinterest (e.g. to estimate noise during own voice pickup or to separateown voice from environment sound, etc.).

The hearing aid may comprise one or more own-voice related algorithms,e.g. a voice control interface and/or a keyword detector. The own-voicerelated algorithms may be optimized to voice from a particular physicaluser or an artificial voice, e.g. using a standard model, e.g. the Headand Torso Simulator (HATS) 4128C from Brüel & Kjær Sound & VibrationMeasurement A/S, or the head and torso model KEMAR from GRAS Sound andVibration A/S, or similar, e.g. a computer model of the acousticpropagation properties of a person. To do so, while still achieving theimprovements of personalized processing, the microphone signal(s) fromthe at least one microphone of the hearing aid may be to pre-weighed(equalized) during signal segments where the own-voice signal dominates(e.g. as estimated using an own-voice detector). In particular, whenoperating the own-voice related algorithms during own-voice activity,the hearing aid (e.g. the signal processor) is configured to weigh theith microphone signal S_(mics,i)(k, l) according toS _(i,modif)(k,l)=d _(HATS,i)(k)/d _(o,i)(k)·S _(mics,i)(k,l),where d_(o,i)(k) is the OVTF of the particular user estimated asdescribed above, d_(HATS,i)(k) is a set of OVTF coefficients as measuredon a HATS or similar physical or artificial model (e.g. offline in asound studio of the HA manufacturer, e.g. estimated as described above)and stored in a memory of (or accessible to) the hearing aid.S_(mics,i)(k, l) denotes a time frequency representation (e.g. providedby a Fourier transform algorithm, e.g. STFT or DFT) of the own-voicesignal recorded on the ith microphone, for the user in question. Therebyown-voice related algorithms of the hearing aid may be optimized toanother voice than the user's own voice (and hence reused for differentusers), while still taking the acoustic propagation properties of theuser's head and body into account.

The hearing aid may comprise one or more own-voice related algorithms,e.g. a voice control interface and/or a speech recognition algorithm,optimized to a voice from a particular physical person or to anartificial or recorded voice from a standard model, wherein—duringsignal segments where the own-voice signal dominates—the at least onemicrophone signal is equalized in dependence of own voice transferfunctions for said particular person or said model and said user,respectively.

The hearing aid may be adapted to provide a frequency dependent gainand/or a level dependent compression and/or a transposition (with orwithout frequency compression) of one or more frequency ranges to one ormore other frequency ranges, e.g. to compensate for a hearing impairmentof a user. The hearing aid comprises a signal processor, which may beconfigured to enhance the input signal(s) and to provide the processedsignal.

The hearing aid may comprise an output unit for providing a stimulusperceived by the user as an acoustic signal based on a processedelectric signal. The output unit may comprise an electrode-array of acochlear implant (for a CI type hearing aid) or a vibrator of a boneconducting hearing aid. The output unit may comprise an outputtransducer. The output transducer may comprise a receiver (loudspeaker)for providing the stimulus as an acoustic signal to the user (e.g. in anacoustic (air conduction based) hearing aid). The output transducer maycomprise a vibrator for providing the stimulus as mechanical vibrationof a skull bone to the user (e.g. in a bone-attached or bone-anchoredhearing aid). The output unit may comprise a synthesis filter bank forconverting a frequency domain signal to a time domain signal. The outputunit may comprise a digital-to-analogue (DA) converter to convert adigital signal to an analogue output signal, e.g. for being presented toa user via an output transducer.

The hearing aid may comprise an input unit for providing the electricinput signal representing sound. The input unit may comprise an inputtransducer, e.g. a microphone, for converting an input sound to anelectric input signal. The input unit may comprise a wireless receiverfor receiving a wireless signal comprising or representing sound (e.g.from the environment around the hearing aid) and for providing anelectric input signal representing said sound. The wireless receiver maye.g. be configured to receive an electromagnetic signal in the radiofrequency range (3 kHz to 300 GHz). The wireless receiver may e.g. beconfigured to receive an electromagnetic signal in a frequency range oflight (e.g. infrared light 300 GHz to 430 THz, or visible light, e.g.430 THz to 770 THz). The input unit may comprise one or more analogue todigital converter(s) (as appropriate) for converting an analogue signalto a digitized signal with a certain sampling rate f_(s). The input unitmay comprise one or more analysis filter banks (as appropriate) forconverting a time domain signal to a frequency domain signal.

The hearing aid may comprise a directional microphone system(beamformer) adapted to spatially filter sounds from the environment,and thereby e.g. enhance (or suppress) a target acoustic source among amultitude of acoustic sources in the local environment of the userwearing the hearing aid. The directional system may be adapted to detect(such as adaptively detect) from which direction a particular part ofthe microphone signal originates. This can be achieved in variousdifferent ways as e.g. described in the prior art. In hearing aids, amicrophone array beamformer is often used for spatially attenuatingbackground noise sources. Many beamformer variants can be found in theliterature. The minimum variance distortionless response (MVDR)beamformer is widely used in microphone array signal processing. Ideallythe MVDR beamformer keeps the signals from the target direction (alsoreferred to as the look direction) unchanged, while attenuating soundsignals from other directions maximally. The generalized sidelobecanceller (GSC) structure is an equivalent representation of the MVDRbeamformer offering computational and numerical advantages over a directimplementation in its original form.

The hearing aid may comprise antenna and transceiver circuitry (e.g. awireless receiver) for wirelessly receiving a direct electric inputsignal from another device, e.g. from a communication device (e.g. asmartphone), a wireless microphone, an entertainment device (e.g. aTV-set), or from another hearing aid. The direct electric input signalmay represent or comprise an audio signal and/or a control signal and/ora status or information signal. The hearing aid may comprisedemodulation circuitry for demodulating the received direct electricinput to provide the direct electric input signal. In general, awireless link established by antenna and transceiver circuitry of thehearing aid can be of any type. The wireless link may be based onnear-field communication, e.g. an inductive link based on an inductivecoupling between antenna coils of transmitter and receiver parts. Thewireless link may be based on far-field, electromagnetic radiation.

The communication between the hearing aid and the other (e.g. auxiliary)device may be in the base band (audio frequency range, e.g. between 0and 20 kHz). Preferably, communication between the hearing aid and theother device is based on some sort of modulation at frequencies above100 kHz. Preferably, frequencies used to establish a communication linkbetween the hearing aid and the other device is below 70 GHz, e.g.located in a range from 50 MHz to 70 GHz, e.g. above 300 MHz, e.g. in anISM range above 300 MHz, e.g. in the 900 MHz range or in the 2.4 GHzrange or in the 5.8 GHz range or in the 60 GHz range (ISM=Industrial,Scientific and Medical, such standardized ranges being e.g. defined bythe International Telecommunication Union, ITU). The wireless link maye.g. be based on a standardized or proprietary technology. The wirelesslink may e.g. be based on Bluetooth technology (e.g. BluetoothLow-Energy technology).

The hearing aid may have a maximum outer dimension of the order of 0.08m or of the order of 0.04 m.

The hearing aid may be or form part of a portable (i.e. configured to bewearable) device, e.g. a device comprising a local energy source, e.g. abattery, e.g. a rechargeable battery. The hearing aid may e.g. be a lowweight, easily wearable, device, e.g. having a total weight less than 20g.

The hearing aid may comprise a forward or signal path between an inputunit (e.g. an input transducer, such as a microphone or a microphonesystem and/or direct electric input (e.g. a wireless receiver)) and anoutput unit, e.g. an output transducer, such as a loudspeaker. Thesignal processor is located in the forward path. The signal processormay be adapted to provide a frequency dependent gain according to auser's particular needs. The hearing aid may comprise an analysis pathcomprising functional components for analyzing the input signal (e.g.determining a level, a modulation, a type of signal, an acousticfeedback estimate, an own voice estimate, a predefined keyword, etc.).Some or all signal processing of the analysis path and/or the signalpath may be conducted in the frequency domain. Some or all signalprocessing of the analysis path and/or the signal path may be conductedin the time domain.

The hearing aid may be configured to convert an analogue electric signalrepresenting an acoustic signal to a digital audio signal in ananalogue-to-digital (AD) conversion process, where the analogue signalis sampled with a predefined sampling frequency or rate f_(s), f_(s)being e.g. in the range from 8 kHz to 48 kHz (adapted to the particularneeds of the application) to provide digital samples x_(n) (or x[n]) atdiscrete points in time t_(n) (or n, n being a time index). Each audiosample may represent the value of the acoustic signal at t_(n) by apredefined number N_(b) of bits, N_(b) being e.g. in the range from 1 to48 bits, e.g. 24 bits. Each audio sample is hence quantized using N_(b)bits (resulting in 2^(Nb) different possible values of the audiosample). A digital sample x has a length in time of 1/f_(s), e.g. 50 μs,for f_(s)=20 kHz. A number of audio samples may be arranged in a timeframe. A time frame may e.g. comprise 64 or 128 (or more) audio datasamples. Other frame lengths may be used depending on the practicalapplication.

The hearing aid may comprise a filter bank comprising an analysis filterbank providing a number of frequency sub-band signals from a time domainsignal and a synthesis filter bank for providing a time domain signalfrom a number of frequency sub-band signals. The hearing aid, e.g. theinput unit, and or the antenna and transceiver circuitry may comprise atime-frequency (TF)-conversion unit for providing a time-frequencyrepresentation of an input signal. The time-frequency representation maycomprise an array or map of corresponding to complex or real values ofthe signal in question in a particular time- and frequency-range. The TFconversion unit may comprise an analysis filter bank for filtering a(time varying) input signal and providing a number of (time varying)output signals each comprising a distinct frequency range of the inputsignal (frequency sub-band signals). The TF conversion unit may comprisea Fourier transformation unit for converting a time variant input signalto a (time variant) signal in the (time-)frequency domain. The frequencyrange considered by the hearing aid from a minimum frequency f_(min) toa maximum frequency f_(max) may comprise a part of the typical humanaudible frequency range from 20 Hz to 20 kHz, e.g. a part of the rangefrom 20 Hz to 12 kHz. Typically, a sample rate f_(s) is larger than orequal to twice the maximum frequency f_(max), f_(s)≥2 f_(max). A signalof the forward and/or analysis path of the hearing aid may be split intoa number NI of frequency bands (e.g. of uniform width), where NI is e.g.larger than 5, such as larger than 10, such as larger than 50, such aslarger than 100, such as larger than 500, at least some of which areprocessed individually. The hearing aid may be adapted to process asignal of the forward and/or analysis path in a number NP of differentfrequency channels (NP≤NI). The frequency channels may be uniform ornon-uniform in width (e.g. increasing in width with frequency),overlapping or non-overlapping.

The hearing aid may be configured to operate in different modes, e.g. anormal mode and one or more specific modes, e.g. selectable by a user,or automatically selectable. A mode of operation may be optimized to aspecific acoustic situation or environment. A mode of operation mayinclude a low-power mode, where functionality of the hearing aid isreduced (e.g. to save power), e.g. to disable wireless communication,and/or to disable specific features of the hearing aid. A mode ofoperation may include a calibration mode, where head-related transferfunctions of the user may be determined according to the presentdisclosure.

The hearing aid may comprise a number of detectors configured to providestatus signals relating to a current physical environment of the hearingaid (e.g. the current acoustic environment), and/or to a current stateof the user wearing the hearing aid, and/or to a current state or modeof operation of the hearing aid. Alternatively, or additionally, one ormore detectors may form part of an external device in communication(e.g. wirelessly) with the hearing aid. An external device may e.g.comprise another hearing aid, a remote control, and audio deliverydevice, a telephone (e.g. a smartphone), an external sensor, etc.

One or more of the number of detectors may operate on the full bandsignal (time domain). One or more of the number of detectors may operateon band split signals ((time-) frequency domain), e.g. in a limitednumber of frequency bands.

The number of detectors may comprise a level detector for estimating acurrent level of a signal of the forward path. The detector may beconfigured to decide whether the current level of a signal of theforward path is above or below a given (L-)threshold value. The leveldetector operates on the full band signal (time domain). The leveldetector operates on band split signals ((time-) frequency domain).

The hearing aid may comprise a voice activity detector (VAD) forestimating whether or not (or with what probability) an input signalcomprises a voice signal (at a given point in time). A voice signal isin the present context taken to include a speech signal from a humanbeing. It may also include other forms of utterances generated by thehuman speech system (e.g. singing). The voice activity detector unit isadapted to classify a current acoustic environment of the user as aVOICE or NO-VOICE environment. This has the advantage that time segmentsof the electric microphone signal comprising human utterances (e.g.speech) in the user's environment can be identified, and thus separatedfrom time segments only (or mainly) comprising other sound sources (e.g.artificially generated noise). The voice activity detector may beadapted to detect as a VOICE also the user's own voice. Alternatively,the voice activity detector may be adapted to exclude a user's own voicefrom the detection of a VOICE.

The hearing aid may comprise an own voice detector for estimatingwhether or not (or with what probability) a given input sound (e.g. avoice, e.g. speech) originates from the voice of the user of the system.A microphone system of the hearing aid may be adapted to be able todifferentiate between a user's own voice and another person's voice andpossibly from NON-voice sounds.

The number of detectors may comprise a movement detector, e.g. anacceleration sensor. The movement detector is configured to detectmovement of the user's facial muscles and/or bones, e.g. due to speechor chewing (e.g. jaw movement) and to provide a detector signalindicative thereof.

The classification unit may be based on or comprise a neural network,e.g. a trained neural network, e.g. a recurrent neural network, e.g. agated recurrent unit (GRU).

The hearing aid may further comprise other relevant functionality forthe application in question, e.g. compression, noise reduction, feedbackcontrol, etc.

The hearing aid may comprise a hearing instrument, e.g. a hearinginstrument adapted for being located at the ear or fully or partially inthe ear canal of a user.

A First Further Hearing Aid

In a further aspect, a hearing aid comprising a data-driven algorithm,e.g. a trained deep neural network (DNN) is provided. The neural networkmay e.g. comprise a recurrent neural network, e.g. a gated recurrentunit (GRU). The data-driven algorithm may be configured to implement anown-voice activity detector (OVAD). Other data-driven own-voice-relevantalgorithms exist, e.g., keyword spotting algorithms, hands-freetelephony related algorithms, etc. Training of the data-driven algorithmis described in section “Application 3. Online Personalization ofOwn-Voice-Driven Algorithms” below.

Features of the hearing aid systems or hearing aids as described above,in the detailed description of embodiments and in the clams may becombined with the first further hearing aid (where appropriate).

A Second Further Hearing Aid

In a further aspect, a hearing aid comprising a signal processor forapplying a number of processing algorithms, possibly includingdata-driven algorithms, e.g. neural network algorithms, e.g. own voiceprocessing algorithms (such as e.g. an own voice detection algorithm, aspeech recognition algorithm, e.g. a keyword detection algorithm, etc.)is provided. The own voice processing algorithms of the hearing aid maybe optimized for the same OVTF, e.g. based on standard model, e.g. ahead and torso model (e.g. HATS or similar models). This is advantageousfor development, debugging, maintenance, and logistics. Personalized ownvoice transfer functions for the user of the hearing aid may be used tomodify (equalize) the microphone signals of the hearing aid, asdescribed in section “Application 4. OVTF Equalization” below.

Features of the hearing aid systems or hearing aids as described above,in the detailed description of embodiments and in the clams may becombined with the second further hearing aid (where appropriate).

A Third Further Hearing Aid

In a further aspect, a hearing aid for playing audio to a user via anoutput transducer of the hearing aid configured to apply head relatedtransfer functions to audio signals either wirelessly received fromanother device or system or internally stored or generated audio signals(e.g. beeps or stored audio or audio generated in the hearing aid)before playing the audio to the user is provided. This has the advantagethat e.g., phone calls, sound notifications, jingles, etc., may beperceived by the user as if they originated from a position outside theuser's body. A combination of a set of measured personal HRTFs with aset of pre-measured HRTFs (e.g., from a model, e.g. HATS), for otherdirections not covered by the personal HRTF set may be used as describedin section “Application 5. Acoustic Rendering using HRTFs” below.

Features of the hearing aid systems or hearing aids as described above,in the detailed description of embodiments and in the clams may becombined with the third further hearing aid (where appropriate).

A Hearing System Comprising a Headset

In a further aspect, a hearing system comprising a headset and anauxiliary device is provided by the present disclosure. The headset isconfigured to be used instead of a hearing aid in the hearing aid systemdescribed above, in the detailed description of embodiments and in theclams.

The headset is adapted for being worn by a user at a or in an ear of theuser. The headset is adapted to establish a communication link to anauxiliary device (e.g. a smartphone) to provide that data can beexchanged between them or forwarded from one to the other. The headsetmay further comprise an input unit comprising at least one microphonefor picking up sound from the environment of the headset, including theuser's own voice, and to provide corresponding at least one electricinput signal representative of said sound, and an output unit, e.g.comprising a loudspeaker, for presenting stimuli perceivable as sound tothe user. The headset may further comprise a signal processor configuredto receive corresponding time segments of the at least one electricinput signal, and at least one auxiliary electric input signal providedby a microphone of said auxiliary device, or corresponding transforms,or selected frequency ranges thereof, and to provide an estimate of anacoustic transfer function from a microphone of said auxiliary device tosaid at least one microphone of the headset. The input unit may comprisetwo or more microphones.

The headset may be configured to (via its input unit) pick up a user'sown voice and transmit it to a far-end communication partner, and toreceive sound from a far-end communication partner and present it to theuser (via the output unit of the headset).

The input unit of the headset may comprise at least two microphones eachproviding an electric input signal. The microphones may be located at orin an ear of the user. The headset may comprise a beamformer filtercomprising one or more beamformers by applying appropriate(predetermined or adaptively determined) filter weights to the at leasttwo electric input signals. The one or more beamformers may comprise anown voice beamformer comprising personalized filter weights, the ownvoice beamformer being configured to enhance signals originating fromthe direction of the user's mouth and to suppress sound signals fromother directions. The personalized filter weights may be determined independence of the estimate of transfer function(s) from said at leastone microphone of said auxiliary device to said at least two microphonesof the headset. The personalized own voice beamformer may be configuredto provide an improved estimate of the user's own voice. The headsetmay—in a communication mode of operation—be configured to transmit theestimate of the user's own voice signal to another device, e.g. to anauxiliary device (e.g. a smartphone).

The auxiliary device, e.g. a smartphone, may comprise at least onemicrophone for picking up sound from the environment of the auxiliarydevice and for providing corresponding at least one auxiliary electricinput signal representative of the sound. The auxiliary device mayfurther comprise a user control interface allowing a user to initiate aspecific calibration mode of operation of the hearing system. Theauxiliary device may further comprise a loudspeaker for playing a testsound. The auxiliary device is adapted to establish a communication linkto the headset to provide that data can be exchanged between them orforwarded from one to the other.

The headset and the auxiliary device may comprise antenna andtransceiver circuitry allowing the communication link to be establishedbetween them.

The headset may comprise a single earpiece adapted to be located at aleft and/or right ear of the user.

The headset may comprise left and right earpieces adapted to be locatedat left and right ears of the user, respectively. The left and rightearpieces may be configured to establish a communication link allowingthe exchange of data between them. The left and right earpieces may eachcomprise respective input and output units. The left and right earpiecesmay each comprise at least two microphones, e.g. located at or in eachof the left and right ears (pinna).

Features of the hearing aid systems and corresponding methods asdescribed above, in the detailed description of embodiments and in theclams may be combined with the present hearing system and headset (whereappropriate).

Use

In an aspect, use of a hearing aid system or a hearing system asdescribed above, in the ‘detailed description of embodiments’ and in theclaims, for determining personalized parameters of a processingalgorithm of the hearing aid or headset. The processing algorithm maye.g. be or comprise a directionality algorithm, e.g. for providing abeamformed signal as a combination of a multitude of electric inputsignals from a multitude of microphones (e.g. from microphones of thehearing aid or headset).

In an aspect, use of a hearing aid or headset as described above, in the‘detailed description of embodiments’ and in the claims, is moreoverprovided. Use may be provided for determining personalized parameters ofa processing algorithm of the hearing aid or headset.

Use may e.g. be provided in a system comprising one or more hearing aids(e.g. hearing instruments), headsets, ear phones, active ear protectionsystems, etc., e.g. in handsfree telephone systems, teleconferencingsystems (e.g. including a speakerphone), public address systems, karaokesystems, classroom amplification systems, etc.

A Method

In an aspect, a method of operating a hearing aid system (or a hearingsystem) is furthermore provided by the present application. The hearingaid system (or hearing system) comprising

-   -   a hearing aid (or headset) adapted for being worn by a user at a        or in an ear of the user, or for being fully or partially        implanted in the head at an ear of the user, and    -   a portable auxiliary device, e.g. a communication device, such        as a smartphone,        wherein the hearing aid system (or the hearing system) is        adapted to establish a communication link between the hearing        aid (or headset) and the auxiliary device to provide that data,        e.g. control signals, status signals, and/or audio signals, can        be exchanged between them or forwarded from one to the other.        The method comprises    -   in the hearing aid (or headset)        -   providing by at least one microphone at least one electric            input signal representative of sound from the environment of            the hearing aid (or headset),        -   processing said at least one electric input signal, or a            signal or signals derived therefrom, and providing a            processed signal,        -   presenting stimuli perceivable as sound to the user            representative of said processed signal,    -   in the auxiliary device        -   providing by at least one microphone at least one auxiliary            electric input signal representative of said sound from the            environment of the hearing aid (or headset),        -   providing a user control interface allowing the user to            initiate a specific calibration mode of operation of the            hearing aid system (or the hearing system).            The method may further comprise: using corresponding time            segments of said at least one electric input signal, and            said at least one auxiliary electric input signal, or            corresponding transforms or selected frequency regions            thereof, to provide an estimate of a transfer function from            said at least one microphone of said auxiliary device to            said at least one microphone of said hearing aid (or            headset).

The method may—in a specific own voice calibration mode ofoperation—comprise to position the auxiliary device in front of themouth of the user. The method may comprise that the at least onemicrophone of the auxiliary device is located less than a maximumdistance from the user's mouth. The maximum distance may e.g. be ≤0.2 mor less than 0.1 m. The method may comprise that the user is instructedabout the location of the auxiliary device via the user controlinterface. The method may comprise that the user is instructed about oneor more of a) the duration, b) the loudness (vocal effort), and c) thecontents and/or type of the utterance that the user is expected toprovide in the own voice calibration mode.

It is intended that some or all of the structural features of thehearing aid systems or hearing aids (or hearing systems or headsets)described above, in the ‘detailed description of embodiments’ or in theclaims can be combined with embodiments of the method, whenappropriately substituted by a corresponding process and vice versa.

Embodiments of the method have the same advantages as the correspondingsystems and devices.

The method may comprise—in said specific calibration mode ofoperation—playing a test sound signal to the environment by theauxiliary device in dependence of an input from said user controlinterface. The method may comprise transmitting the test sound signalfrom the auxiliary device to the hearing aid (or headset).

A Computer Readable Medium or Data Carrier

In an aspect, a tangible computer-readable medium (a data carrier)storing a computer program comprising program code means (instructions)for causing a data processing system (a computer) to perform (carry out)at least some (such as a majority or all) of the (steps of the) methoddescribed above, in the ‘detailed description of embodiments’ and in theclaims, when said computer program is executed on the data processingsystem is furthermore provided by the present application.

By way of example, and not limitation, such computer-readable media cancomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othermedium that can be used to carry or store desired program code in theform of instructions or data structures and that can be accessed by acomputer. Disk and disc, as used herein, includes compact disc (CD),laser disc, optical disc, digital versatile disc (DVD), floppy disk andBlu-ray disc where disks usually reproduce data magnetically, whilediscs reproduce data optically with lasers. Other storage media includestorage in DNA (e.g. in synthesized DNA strands). Combinations of theabove should also be included within the scope of computer-readablemedia. In addition to being stored on a tangible medium, the computerprogram can also be transmitted via a transmission medium such as awired or wireless link or a network, e.g. the Internet, and loaded intoa data processing system for being executed at a location different fromthat of the tangible medium.

A Computer Program

A computer program (product) comprising instructions which, when theprogram is executed by a computer, cause the computer to carry out(steps of) the method described above, in the ‘detailed description ofembodiments’ and in the claims is furthermore provided by the presentapplication.

A Data Processing System

In an aspect, a data processing system comprising a processor andprogram code means for causing the processor to perform at least some(such as a majority or all) of the steps of the method described above,in the ‘detailed description of embodiments’ and in the claims isfurthermore provided by the present application.

An APP

In a further aspect, a non-transitory application, termed an APP, isfurthermore provided by the present disclosure. The APP comprisesexecutable instructions configured to be executed on an auxiliary deviceto implement a user interface for a hearing aid or a hearing aid system(or a headset or a hearing system) described above in the ‘detaileddescription of embodiments’, and in the claims. The APP may beconfigured to run on a cellular phone, e.g. a smartphone, or on anotherportable device allowing communication with said hearing aid or saidhearing aid system.

The user interface may be configured to allow a user to controlfunctionality of the hearing aid system, including an initiation of aspecific calibration mode of operation of the hearing aid system orhearing aid.

The hearing aid system (including the APP) may be configured to allow auser to initiate a specific calibration mode of operation of the hearingaid system via the user interface. The hearing aid system (including theAPP) may be configured to instruct the user via the user interface ofthe auxiliary device how to position the auxiliary device relative tothe user in dependence of the selected calibration mode. In the ownvoice calibration mode, the user interface may be configured to instructthe user to position the auxiliary device so that the at least onemicrophone of the auxiliary device is located next to the mouth of theuser. In the own voice calibration mode, the hearing aid system(including the APP) may be configured to instruct the user via the userinterface of the auxiliary device to speak a certain phrase or sentence(e.g. with a particular vocal effort, e.g. loud or soft, etc., e.g. independence of a current noise level around the user) or for a specificor minimum duration. In the far-field calibration mode, the userinterface may be configured to instruct the user to position theauxiliary device at a preferred location relative to the user(specifically to the hearing aid microphone(s) for which an (acoustic)transfer function is to be estimated), e.g. held in a hand, or on astick, or located at or on a table or other support.

Definitions

In the present context, a hearing aid, e.g. a hearing instrument, refersto a device, which is adapted to improve, augment and/or protect thehearing capability of a user by receiving acoustic signals from theuser's surroundings, generating corresponding audio signals, possiblymodifying the audio signals and providing the possibly modified audiosignals as audible signals to at least one of the user's ears. Suchaudible signals may e.g. be provided in the form of acoustic signalsradiated into the user's outer ears, acoustic signals transferred asmechanical vibrations to the user's inner ears through the bonestructure of the user's head and/or through parts of the middle ear aswell as electric signals transferred directly or indirectly to thecochlear nerve of the user.

The hearing aid may be configured to be worn in any known way, e.g. as aunit arranged behind the ear with a tube leading radiated acousticsignals into the ear canal or with an output transducer, e.g. aloudspeaker, arranged close to or in the ear canal, as a unit entirelyor partly arranged in the pinna and/or in the ear canal, as a unit, e.g.a vibrator, attached to a fixture implanted into the skull bone, as anattachable, or entirely or partly implanted, unit, etc. The hearing aidmay comprise a single unit or several units communicating (e.g.acoustically, electrically or optically) with each other. Theloudspeaker may be arranged in a housing together with other componentsof the hearing aid, or may be an external unit in itself (possibly incombination with a flexible guiding element, e.g. a dome-like element).

More generally, a hearing aid comprises an input transducer forreceiving an acoustic signal from a user's surroundings and providing acorresponding input audio signal and/or a receiver for electronically(i.e. wired or wirelessly) receiving an input audio signal, a (typicallyconfigurable) signal processing circuit (e.g. a signal processor, e.g.comprising a configurable (programmable) processor, e.g. a digitalsignal processor) for processing the input audio signal and an outputunit for providing an audible signal to the user in dependence on theprocessed audio signal. The signal processor may be adapted to processthe input signal in the time domain or in a number of frequency bands.In some hearing aids, an amplifier and/or compressor may constitute thesignal processing circuit. The signal processing circuit typicallycomprises one or more (integrated or separate) memory elements forexecuting programs and/or for storing parameters used (or potentiallyused) in the processing and/or for storing information relevant for thefunction of the hearing aid and/or for storing information (e.g.processed information, e.g. provided by the signal processing circuit),e.g. for use in connection with an interface to a user and/or aninterface to a programming device. In some hearing aids, the output unitmay comprise an output transducer, such as e.g. a loudspeaker forproviding an air-borne acoustic signal or a vibrator for providing astructure-borne or liquid-borne acoustic signal. In some hearing aids,the output unit may comprise one or more output electrodes for providingelectric signals (e.g. to a multi-electrode array) for electricallystimulating the cochlear nerve (cochlear implant type hearing aid).

In some hearing aids, the vibrator may be adapted to provide astructure-borne acoustic signal transcutaneously or percutaneously tothe skull bone. In some hearing aids, the vibrator may be implanted inthe middle ear and/or in the inner ear. In some hearing aids, thevibrator may be adapted to provide a structure-borne acoustic signal toa middle-ear bone and/or to the cochlea. In some hearing aids, thevibrator may be adapted to provide a liquid-borne acoustic signal to thecochlear liquid, e.g. through the oval window. In some hearing aids, theoutput electrodes may be implanted in the cochlea or on the inside ofthe skull bone and may be adapted to provide the electric signals to thehair cells of the cochlea, to one or more hearing nerves, to theauditory brainstem, to the auditory midbrain, to the auditory cortexand/or to other parts of the cerebral cortex.

A hearing aid may be adapted to a particular user's needs, e.g. ahearing impairment. A configurable signal processing circuit of thehearing aid may be adapted to apply a frequency and level dependentcompressive amplification of an input signal. A customized frequency andlevel dependent gain (amplification or compression) may be determined ina fitting process by a fitting system based on a user's hearing data,e.g. an audiogram, using a fitting rationale (e.g. adapted to speech).The frequency and level dependent gain may e.g. be embodied inprocessing parameters, e.g. uploaded to the hearing aid via an interfaceto a programming device (fitting system), and used by a processingalgorithm executed by the configurable signal processing circuit of thehearing aid.

A ‘hearing aid system’ refers to a system comprising one or two hearingaids, and a ‘binaural hearing aid system’ refers to a system comprisingtwo hearing aids and being adapted to cooperatively provide audiblesignals to both of the user's ears. Hearing aid systems or binauralhearing aid systems may further comprise one or more ‘auxiliarydevices’, which communicate with the hearing aid(s) and affect and/orbenefit from the function of the hearing aid(s). Such auxiliary devicesmay include at least one of a remote control, a remote microphone, anaudio gateway device, an entertainment device, e.g. a music player, awireless communication device, e.g. a mobile phone (such as asmartphone) or a tablet or another device, e.g. comprising a graphicalinterface. Hearing aids, hearing aid systems or binaural hearing aidsystems may e.g. be used for compensating for a hearing-impairedperson's loss of hearing capability, augmenting or protecting anormal-hearing person's hearing capability and/or conveying electronicaudio signals to a person. Hearing aids or hearing aid systems may e.g.form part of or interact with public-address systems, active earprotection systems, handsfree telephone systems, car audio systems,entertainment (e.g. TV, music playing or karaoke) systems,teleconferencing systems, classroom amplification systems, etc.

Embodiments of the disclosure may e.g. be useful in applications such asbeamforming, handsfree telephony, voice control, keyword spotting, etc.

BRIEF DESCRIPTION OF DRAWINGS

The aspects of the disclosure may be best understood from the followingdetailed description taken in conjunction with the accompanying figures.The figures are schematic and simplified for clarity, and they just showdetails to improve the understanding of the claims, while other detailsare left out. Throughout, the same reference numerals are used foridentical or corresponding parts. The individual features of each aspectmay each be combined with any or all features of the other aspects.These and other aspects, features and/or technical effect will beapparent from and elucidated with reference to the illustrationsdescribed hereinafter in which:

FIG. 1A illustrates a procedure for estimating own voice transferfunctions (OVTFs) using a hearing aid system comprising a hearing aidand an auxiliary device, e.g. a mobile phone, during a calibration modeof the hearing aid system, wherein the user's own-voice signal s_(ov)(n)is picked up by a microphone of the mobile phone and by microphones inthe hearing aid (signals s_(i)(n)), and wherein the signals are used forestimating OVTFs H_(ov,i)(ω) (and relative OVFTs d_(ov,i)(ω)), and

FIG. 1B illustrates a time dependence of an own voice control signalduring a calibration mode from a start time t_(start) to a stop timet_(stop),

FIG. 2 illustrates a procedure for estimating (frontal) HRTFs using anauxiliary device, e.g. a mobile phone, wherein a test sound signals_(f)(n) is emitted from a loudspeaker of the mobile phone, and theresulting sound signals s_(i)(n) are picked up by the HA microphones,

FIG. 3 shows a hearing aid system according to an embodiment of thepresent disclosure, and

FIGS. 4A, 4B, and 4C together illustrate an exemplary applicationscenario of an embodiment of a hearing system according to the presentdisclosure, wherein

FIG. 4A illustrates a user, a binaural hearing aid system and anauxiliary device during a calibration procedure of the noise reductionsystem,

FIG. 4B illustrates the auxiliary device running an APP for initiatingthe calibration procedure for personalizing own voice transferfunctions, and

FIG. 4C illustrates the auxiliary device running an APP for initiatingthe calibration procedure for personalizing head related transferfunctions, and

FIG. 5 shows an embodiment of a headset or a hearing aid comprising ownvoice estimation and the option of transmitting the own voice estimateto another device, and to receive sound from another device forpresentation to the user via a loudspeaker, e.g. mixed with sound fromthe environment of the user.

The figures are schematic and simplified for clarity, and they just showdetails which are essential to the understanding of the disclosure,while other details are left out. Throughout, the same reference signsare used for identical or corresponding parts.

Further scope of applicability of the present disclosure will becomeapparent from the detailed description given hereinafter. However, itshould be understood that the detailed description and specificexamples, while indicating preferred embodiments of the disclosure, aregiven by way of illustration only. Other embodiments may become apparentto those skilled in the art from the following detailed description.

DETAILED DESCRIPTION OF EMBODIMENTS

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations. Thedetailed description includes specific details for the purpose ofproviding a thorough understanding of various concepts. However, it willbe apparent to those skilled in the art that these concepts may bepracticed without these specific details. Several aspects of theapparatus and methods are described by various blocks, functional units,modules, components, circuits, steps, processes, algorithms, etc.(collectively referred to as “elements”). Depending upon particularapplication, design constraints or other reasons, these elements may beimplemented using electronic hardware, computer program, or anycombination thereof.

The electronic hardware may include micro-electronic-mechanical systems(MEMS), integrated circuits (e.g. application specific),microprocessors, microcontrollers, digital signal processors (DSPs),field programmable gate arrays (FPGAs), programmable logic devices(PLDs), gated logic, discrete hardware circuits, printed circuit boards(PCB) (e.g. flexible PCBs), and other suitable hardware configured toperform the various functionality described throughout this disclosure,e.g. sensors, e.g. for sensing and/or registering physical properties ofthe environment, the device, the user, etc. Computer program shall beconstrued broadly to mean instructions, instruction sets, code, codesegments, program code, programs, subprograms, software modules,applications, software applications, software packages, routines,subroutines, objects, executables, threads of execution, procedures,functions, etc., whether referred to as software, firmware, middleware,microcode, hardware description language, or otherwise.

The present application relates to the field of hearing aids. It dealsin particular to various aspects of retrieval and/or detection of ahearing aid user's own voice, e.g. in relation to beamforming and/orpreservation or reestablishment of spatial cues.

Personal own voice transfer functions (OVTFs) may be estimated simply byusing a portable electronic device, e.g. a mobile phone (or similarcommunication device comprising a microphone and a transmitter), or awireless microphone. Imagine that the hearing aid (HA) system is in anOVTF estimation mode (calibration mode), e.g., triggered by the HA-useror a hearing care professional (HCP), e.g. via a user interface, e.g. anAPP (e.g. of a mobile phone). In such calibration mode, the hearing aidsystem may be configured to prompt the HA user to place the mobile phonein front of his/her mouth and speak in a natural manner for some time,e.g. 1-10 seconds. For simplicity, the user may be asked to speakparticular sound elements, e.g. a particular sentence (e.g. presented atthe user interface, e.g. with a certain vocal effort, e.g. dependent onan environment noise level). For OVTF estimation the exact content ofthe speech signal is irrelevant. The OVTF estimation procedure shouldpreferably take place in an otherwise acoustically quiet situation. Thismay be verified by the HA-system, e.g. the hearing aid(s), or the mobilephone, or a combination of both (or by a separate device), beforeinitiating the estimation (calibration) procedure. Ideally, the usershould be located away from reflecting surfaces, such as walls, etc.,during calibration. Furthermore, ideally, the auxiliary device (e.g. amobile phone) should be placed in a manner to reduce reflections fromthe phone surface to the microphones of the HA (e.g. by positioning itto have its largest surface, e.g. its display, in a horizontal plane,when the user is upright, cf. FIG. 1A).

FIG. 1A illustrates a procedure for estimating own voice transferfunctions (OVTFs) using a hearing aid system according to the presentdisclosure. The hearing aid system comprises a hearing aid (HD) and anauxiliary device (AD), e.g. a mobile phone. FIG. 1A illustrates acalibration mode of operation of the hearing aid system, wherein theuser's own-voice signal s_(ov)(n) is picked up by a microphone (ADM) ofthe mobile phone (AD) and by microphones (M₁, M₂, M₃) in the hearing aid(providing signals s_(i)(n), i=1, 2, 3, and wherein the signals are usedfor estimating frequency dependent OVTFs H_(ov,i)(ω) (i=1, 2, 3) fromthe HA-users' mouth (actually from the microphone of the phone (AD)) tothe microphones of the HA-system (and possibly relative OVFTsd_(ov,i)(ω)), where co denotes (angular) frequency (2πf, where f isfrequency). The hearing aid of FIG. 1A comprises a BTE-part adapted forbeing positioned at or behind pinna, and an ITE part adapted for beingpositioned at or in the user's ear canal. The two parts are connected bya connecting element (IC), e.g. an acoustic tube or an electric cable(or both).

The speech signal of the HA-user is picked up by the microphone(s) (ADM)in the phone (AD) and by the microphone(s) (M_(i)) in the users' HA(s)(HD). From these signals, the acoustic transfer function from theHA-users' mouth (actually from the microphone of the phone) to themicrophones of the HA-system may be estimated. The user may wear ahearing aid at one ear or at both ears.

In more detail, let s_(ov)(n) denote the own-voice time-domain signalpicked up by a microphone in the mobile phone, placed at the mouthreference point, i.e., a position in front of (and close to) theHA-users' mouth. Furthermore, let s₁(n), . . . , s_(M)(n) denote thecorresponding speech signals picked up by the M microphones of the HA(either in one HA at one ear, or in two HAs at both ears or inadditional devices, e.g., a separate wireless microphone). Consider theFourier transform of the picked-up signals and denote them by S_(ov)(ω)and S₁(ω), . . . , S_(M)(ω), respectively. Clearly, the acoustictransfer function from the mouth reference point to microphone i, i.e.,the OVTF, is given by

${H_{o,i}(\omega)} = {\frac{S_{i}(\omega)}{S_{ov}(\omega)}.}$

In practice, S_(i)(ω) and S_(ov)(ω) are found by applying the DiscreteFourier Transform (DFT) to the microphone signals s_(ov)(n) and s₁(n), .. . , s_(M)(n) leading to discrete acoustic transfer functions

${{H_{{ov},i}(k)} = \frac{S_{i}(k)}{S_{o\nu}(k)}},{k = 0},\ldots\mspace{14mu},{K - 1},$where k is the frequency bin index and K is the order of the DFT, e.g.64 or 128.

For signal processing applications, it is often useful to collect theOVTFs for all microphones in one vector,H _(ov)(k)=[H _(ov,1)(k) . . . H _(ov,M)(k)]^(T).

It is often of relevance (see examples below) to consider relativeOVTFs, defined as

${{d_{{o\nu},i}(k)} = \frac{H_{{ov},i}(k)}{H_{{ov},i^{\prime}}(k)}},{k = 0},\ldots\mspace{14mu},{K - 1},$where 1≤i′≤M is the index of a pre-selected reference microphone (one ofthe microphones in the HA system, e.g. a front microphone of a hearingaid), and to collect these in a relative OVTF vector, defined asd _(ov)(k)=[d _(ov,1)(k) . . . d _(ov,M)(k)]^(T).

In summary, OVTFs H_(ov)(k)=[H_(ov,1)(k) . . . H_(ov,M)(k)]^(T) andrelative OVTFs d_(ov)(k)=[d_(ov,1)(k) . . . d_(ov,M)(k)]^(T) may beestimated from microphone signals s_(ov)(n) and s₁(n), . . . , s_(M)(n).Note, when estimated in the manner described here, these OVTFs arepersonal, i.e., they reflect the personal acoustics (head shape, size,pinna, HA-location) of a particular HA-user. In practice, slightly moreadvanced, noise-robust, and data-efficient methods may be applied forestimating the OFTFs H_(ov,i)(k) [Farina, 2000] rather than simplyforming the ratio H_(ov,i)(k)=S_(i)(k)/S_(ov)(k). The estimationprocedure described above assumes that all relevant signals areavailable for processing in one place—so we assume that the relevantsignals are transmitted (e.g. wirelessly), e.g. from the mobile phone tothe hearing aid system (or elsewhere).

FIG. 1B illustrates a time dependence (Time) of an own voice controlsignal (OVD) during a calibration mode from a start time t_(start) to astop time t_(stop). The own voice control signal is equal to 1indicating a presence of the user' own voice (or a presence with aprobability above a certain (e.g. first) threshold) in a time rangebetween t₁ and t₂ within the calibration period (between t_(start) andt_(stop)) and equal to 0 indicating an absence of the user' own voice(or a presence with a probability below a certain (e.g. second)threshold) in the calibration period outside the time range [t₁; t₂].

Similarly, it is of interest to estimate the (relative) acoustictransfer function from the typical position of a conversation partner(or a competing speaker) to the microphones of the HA—we denote thisacoustic transfer function as the frontal head-related transfer function(HRTF). Estimation of this HRTF may be done using a mobile phone as awireless loudspeaker. EP2928215A1 describes the use of an auxiliarydevice (e.g. a mobile telephone) for self-calibration of beamformers forretrieving non-own-voice sound sources of interest.

Imagine that the HA system is in a (frontal) HRTF estimation mode, e.g.triggered by the HA-user or a hearing care professional (HCP) via anAPP. The user holds the mobile phone in a frontal position at an arm'slength distance (the typical position of a conversation partner) at aheight corresponding to the users' mouth, the loudspeaker of the mobiletelephone emits a test sound signal s_(f)(n) from its speaker, and theprobe signal is picked up by the microphones of the HA-system worn bythe user (cf. FIG. 2).

FIG. 2 illustrates a procedure for estimating (frontal) HRTFs using anauxiliary device (AD), e.g. a mobile phone, wherein a test sound signal(‘Test sound’, s_(f)(n)) is emitted from a loudspeaker (AD-SPK) of themobile phone (AD) and the resulting sound signals (s_(i)(n)) are pickedup by the HA microphones (M_(i), i=1, 2, 3). Based on the emitted andreceived signals, acoustic transfer functions H_(f,i)(ω) (i=1, 2, 3) (orcorresponding impulse responses h_(f,i)(n)) may be estimated.

A camera of the mobile phone may be used to give feedback to the user,that the mobile phone is in the correct position (e.g. according to apredefined criterion). The duration of the test sound signal could beranging from a few 100 ms to several seconds (e.g. in the range between1 s and 10 s; the longer the duration, the more accurately the HRTF maybe estimated, but the higher the risk that the user is unable to holdthe mobile phone or his or her head still). The exact content of thetest sound signal is less important, as long as the signal containsenergy at all relevant frequencies (e.g. speech frequencies). Ideally,the estimation procedure takes place in an otherwise acoustically quietsituation and in a room without too many reflections, e.g. in a roomwith soft carpets, curtains, etc. Even if the measurement takes place ina reflective environment, the late reflections may be removed from theestimated impulse response (IR) by truncation of the ‘reverberant’ IRtail.

In an embodiment the phone is mounted in a selfie stick. Based on acorrelation (e.g. estimated by the hearing aid system, e.g. the hearingaid or the auxiliary device) between the hearing aid microphones and themicrophone of the mobile phone, the length of the selfie stick may beadjusted such that a desired distance between the hearing instrumentmicrophones and the phone in front of the user is obtained.

In the setup of FIG. 2, the distance between the hearing aid (HD, andits microphones, M₁, M₂, M₃) and the auxiliary device (AD, and itsloudspeaker, AD-SPK) is indicated (L). It may e.g. be determined by adistance sensor, e.g. a light sensor, e.g. a camera, e.g. located in theauxiliary device) or the by (a predetermined) length of an arm or aselfie-stick). The hearing aid (HD) may be of any style, e.g., as shownin FIG. 2, comprising a BTE-part adapted for being located behind an ear(pinna) of the user and an ITE-part (ITE) adapted for being located ator in an ear canal of the user. The two parts are connected (e.g.mechanically (e.g. acoustically) and/or electrically) by aninterconnecting element (IC). In the embodiment of FIG. 2, the BTE-partcomprises three microphones (M₁, M₂, M₃), but other numbers mayberelevant. Two of the three microphones are (intended to be) located in ahorizontal plane when the user is in an upright position to allowbeamforming in an expected direction of a communication partner. Byhaving a microphone located outside the horizontal plane, more optionsfor beamforming are provided, e.g. in a direction of the user's mouth,e.g. for picking up the voice of the user.

The user may (e.g. via the user interface, e.g. via the auxiliarydevice) initiate the (calibration) measurement, when the auxiliarydevice is located in an intended position relative to the user. Themeasurement may also be initiated when a certain distance is obtained(as e.g. determined by a distance sensor). Hereby the user does not haveto actively initiate the measurement.

In an embodiment, the user is notified prior to the beginning of themeasurement (to achieve that the user is not moving during themeasurement). Notification may happen via the phone screen, by audiofrom the phone, or via audio played via the output unit of the hearingaid. This has the advantage that the user becomes aware not to move.

As before, let s₁(n), . . . , s_(M)(n) denote the corresponding signalspicked up by the microphones of the HA-system. Now the frontal HRTFH_(f,i)(k) from the mobile phone to the ith microphone, and the frontalrelative HRTF d_(f,i)(k)=H_(f,i)(k)/H_(f,i′)(k) can be estimated exactlyas in the discussion in connection with FIG. 1A, 1B, but where theown-voice signal s_(o)(n) is replaced by the test sound signal s_(f)(n)generated by the mobile phone. The resulting frontal HRTF vector isdenoted asH _(f)(k)=[H _(f,1)(k) . . . H _(f,M)(k)]^(T),and the relative frontal HRTF is denoted asd _(f)(k)=[d _(f,1)(k) . . . d _(f,M)(k)]^(T).

In practice, the (relative) HRTF may be estimated using slightly morecomplicated procedures than described in the previous section.Specifically, it may be beneficial that the test sound signal is a chirpsignal (a tonal signal whose frequency increases with time); in thiscase, the HRTF may be estimated using the procedure outlined in [Farina,2000].

The HRTFs may be measured for multiple sound source positions (angles),not only the frontal. Clearly, it is hard for a person to hold a mobilephone in her hand at an angle of, say, 25 degrees wrt. his/her nosedirection. However, the hearing aid system may be configured to providethat the auxiliary device (e.g. the phone) delivers feedback to the user(e.g., via the loudspeaker or the screen) if/when the phone is held inthe correct position. This may be achieved using the camera of phone(e.g. based on a user input regarding the position of interest, e.g.selected among a number of predefined positions, e.g. via the userinterface). Once in the correct position, the phone emits the test soundsignal and measures the HRTF as described above. This process could berepeated for a range of front-half-plane locations of the mobile phone.

EXAMPLES Application 1. Personalized Own-Voice Beamformer/NoiseReduction System

This application uses the OVTFs d_(o)(k)=[d_(o,1)(k) . . .d_(o,M)(k)]^(T) estimated as described above.

For an application such as handsfree telephony in HAs andvoice-controlled HAs, it is essential to be able to retrieve (anestimate of) a clean version of the users' speech signal, even inacoustically noisy situations. In order to do so, one can designbeamforming systems based on the microphone signals of the HA system inorder to enhance signals originating from the direction of the users'mouth and suppress sound signals from other directions.

For example, it is well-known that the filter coefficients of a MinimumVariance Distortion-Less Response (MVDR) beamformer are given by

${{w\left( {k,l} \right)} = \frac{{C_{v}^{- 1}\left( {k,l} \right)}{d(k)}}{{d^{H}(k)}{C_{v}^{- 1}\left( {k,l} \right)}{d(k)}}},$where C_(v)(k, l) denotes the cross-power spectral density matrix atfrequency k and time instant l (see e.g. [Jensen et al., 2015] and thereferences therein for methods for estimating C_(v)(k, l), and whered(k) is the relative acoustic transfer function from a sound source ofinterest to microphones providing input to the MVDR-beamformer.

Inserting the estimated OVTF vector, d_(o)(k), into this expressionleads to a personalized own voice beamformer,

${{w_{o}\left( {k,l} \right)} = \frac{{C_{v}^{- 1}\left( {k,l} \right)}{d_{o}(k)}}{{d_{o}^{H}(k)}{C_{v}^{- 1}\left( {k,l} \right)}{d_{o}(k)}}},$which leads to a better own-voice retrieval/noise reduction trade-offthan when using a non-personalized d(k), e.g. as estimated from aHead-And-Torso Simulator (HATS). Alternative own-voice retrieval systemseasily follow, e.g. based on the Multi-Channel Wiener Filter,Delay-and-Sum Beamformer [Brandstein et al., 2001], Beamformer-InformedPostfilter solutions [Jensen et al., 2015], etc.

Application 2. Personalized Own-Voice Beamformer with FrontalInterference Rejection

This application uses the OVTFs d_(o)(k)=[d_(o,1)(k) . . .d_(o,M)(k)]^(T) estimated as described above, together with the frontalHRTFs d_(f)(k)=[d_(f,1)(k) . . . d_(f,M)(k)]^(T) estimated as describedabove.

The idea is an extension of the idea described in section ‘Application1’ above, where, in addition to retrieving the users' own voice signal,a spatial null is directed towards the frontal direction, in order tomaximally suppress a presumed competing speaker. It is well-known that abeamformer, which can perform this task is a special case of a LinearConstrained Minimum Variance (LCMV) beamformer. The beamformercoefficient vector is found by solving the problem

$\min\limits_{w}{{w^{H}\left( {k,1} \right)}{C_{v}\left( {k,l} \right)}{w\left( {k,1} \right)}}$subject to the constraintsw ^(H)(k,l)d _(o)(k)=1,andw ^(H)(k,l)d _(f)(k)=0.

It is well-known that this problem obeys a simple, closed-form solution[Haykin, 2001].

Alternatives to the LCMV beamformer solution exist—for example, it isstraightforward to extend it with a postfilter.

Application 3. Online Personalization of Own-Voice-Driven Algorithms

This application uses the OVTFs d_(o)(k)=[d_(o,1)(k) . . .d_(o,M)(k)]^(T) estimated as described above and assumes (optionally)that a batch of the users' own voice is recorded with theHA-microphones. An extension of the idea also uses the (frontal) HRTFd_(f)(k)=[d_(f,1)(k) . . . d_(f,M)(k)]^(T) estimated as described above.

Assume that a data-driven algorithm is present in the HA-system. Suchalgorithm could typically involve a deep neural network (DNN) trained tosolve a relevant task. In the example below, we assume that thisalgorithm is an own-voice activity detector (OVAD), but this is only anexample—other data-driven own-voice-relevant algorithms exist, e.g.,keyword spotting algorithms, hands-free telephony related algorithms,etc.

Assume, for example, that the OVAD is based on a deep neural network(DNN), which is trained to classify each time-frequency tile in theinput signal as a) own-voice dominated, b) not own-voice dominated(comprising background noise, external talkers, silence, etc.), cf. e.g.[Garde, 2019]. An OVAD serves as a pre-requisite for other algorithms,e.g., algorithms for estimating the noise cross-power spectral densitymatrix C_(v)(k, l), etc., cf. e.g. [Garde, 2019]. Traditionally, thetraining of such DNN-OVAD takes place off-line, i.e., prior to HA-usage,using speech signals uttered by many different speakers (males, females,children) and recorded by HAs on their individual ears. The resultingOVAD-algorithm works well on average across a group of representativeusers—this is a speaker-independent algorithm.

However, given access to the personal OVTF d_(o)(k) along with examplesof speech from the user in question, the DNN may be re-trained (ortrained further, aka transfer learning) online, i.e., during HA usage,using artificially generated own-voice microphone signals. Specifically,the artificial own-voice signals may be generated according toS _(i)(k,l)=d _(o,i)(k)·S _(o)(k,l),where S_(i)(k, l) is the Short-Time Fourier Transform of the artificialpersonalized own-voice signal recorded at microphone i, d_(o,i)(k) isthe OVTF estimated as described above, and S_(o)(k, l) is the STFT ofthe recording of the users' own voice. Time-domain versions of theartificial own-voice microphone signals may be constructed by applyingthe inverse STFT to the STFT-signals. If a recording of the users' ownvoice is not available, a collection of other speech signals may beused, e.g. from speakers of the same gender as the user, if suchinformation is available. In this situation, the data-driven algorithmwill be personalized in terms of OVTFs but not in terms of the users'voice characteristics.

Re-training (or continued training) of a DNN during HA-usage may be harddue to memory and computational complexity limitations of the HA. Onecould bypass this problem by transmitting the relevant data (OVTFs andoptional own voice signals and optional DNN parameters) wirelessly to anexternal computational unit, which, after re-training, would transmitthe resulting DNN weights back to the HA-system.

As already mentioned, the presented idea of using the OVTFs and(optionally) recordings of the users' own voice is not limited to theOVAD example described above, but may be applied to personalize anydata-driven algorithm onboard the HA.

An extension of the idea involves including a frontal competing speakerin the artificially generated training data. In particular, noisyown-voice signals may be generated according toX _(i)(k,l)=d _(o,i)(k)·S _(o)(k,l)+d _(f,i)(k)·S _(f)(k,l)+V(k,l),where d_(f,i)(k) are (frontal) HRTFs, e.g. measured as described in Sec.2.2, S_(f)(k, l) is the STFT of the voice signal of a competing speaker,and V(k, l) is an arbitrary noise signal representing non-coherent noisesources in the acoustic environment. The competing speech signalS_(f)(k, l) could be generated from arbitrary speech signals from alarge quantity of male and female speakers (as the competing speaker isgenerally unknown in practice), and V(k, l) could be generated fromrelevant acoustic noise, e.g., noise from a cafeteria situation or apassenger-in-a-train situations, etc. as recorded by the HA-microphoneson a HATS. It is assumed that signals S_(f)(k, l), and V(k, l) arepresent in an external computational device, where (re-)training of thenetwork weights take place.

Application 4. OVTF Equalization

The idea uses the OVTFs d_(o)(k)=[d_(o,1)(k) . . . d_(o,M)(k)]^(T)estimated as described above.

One approach to realize personalized own-voice processing is bymodifying the actual signal processing algorithms taking place in theHA-system, e.g. (re-)training DNN weights to fit personal head acoustics(example 3) or modifying beamformer weights to reflect personal head-and torso-acoustics. It may, however, be desirable to maintain the samesignal processing algorithm implementations (including DNN weights) forall users (such processing algorithms may include own-voice-relevantalgorithms, e.g. an own voice detection algorithm, a speech recognitionalgorithm, e.g. a keyword detection algorithm, etc.). In particular, itwould be desirable, if the own voice processing algorithms on-board theHA system were optimized for the same OVTF, e.g. the one of a HATS—thiswould make system development, debugging, maintenance, and logisticseasier.

To do so, while still achieving the improvements of personalizedprocessing, we propose to pre-weigh or equalize the microphone signalsduring signal regions where the own-voice signal dominates (e.g. asestimated using an OVAD). In particular, when operating the own-voicerelated algorithms during own-voice activity, we propose to weigh theith microphone signal S_(mics,i)(k, l) according toS _(i,modif)(k,l)=d _(HATS,i)(k)/d _(o,i)(k)·S _(mics,i)(k,l),where d_(o,i)(k) is the OVTF of the particular user estimated asdescribed above, d_(HATS,i)(k) is a set of OVTF coefficients as measuredon a HATS (offline in a sound studio of the HA manufacturer, e.g.estimated as described above) and stored in the HA memory, andS_(mics,i)(k, l) denotes the STFT of the own-voice signal recorded onthe ith microphone, for the user in question.

The proposed equalization scheme transforms the own-voice microphonesignals of a particular user, to the own-voice microphone signals of aHATS. This allows the subsequent processing applied in the HA-system tobe optimized for a HATS, irrespective of the actual user. In otherwords, the processing after the equalization would be identical for allusers.

Application 5. Acoustic Rendering Using HRTFs

The idea uses the (frontal) absolute HRTF H_(f)(k)=[H_(f,1)(k) . . .H_(f,M)(k)]^(T), estimated as described above. Optionally, the idea usesthe frontal HRTF in addition to absolute HRTFs measured from otherdirections than the frontal.

We propose to combine the set of measured personal HRTFs with a set ofpre-measured HRTFs (e.g., from a HATS), for other directions not coveredby the personal HRTF set. We propose to use the combined set of HRTFsfor spatially realistic rendering of acoustic signals for the user of ahearing device. In particular, the combined HRTF set makes it possibleto play back sounds of interest for the user, e.g., phone calls, soundnotifications, jingles, etc., as if they originated from a positionoutside the users body, e.g., in the frontal position, or slightly tothe left, etc., or to render an ambient signal more realistically, usingmore or all HRTFs in the combined set.

Specifically, without loss of generality, let i=1 denote the index of aHA-microphone close to the left eardrum of the user, and let i=2 denotethe index of a HA-microphone close to the right eardrum of the user.Also, still without loss of generality, let us consider rendering asound source as originating from the frontal position (for example).Hence, H_(f,1)(k) denotes the acoustic transfer function from a positionin front of the user to her left ear, while H_(f,2)(k) denotes theacoustic transfer function from the same position in front of the userto her right ear.

Then a sound of interest for the user may be rendered as originatingfrom the front according toS _(i)(k,l)=H _(f,i)(k)S(k,l),i=1,2,where S(k, l) is the STFT of the sound of interest, while S₁(k, l) andS₂(k, l) is the STFT of the signal present to the left and right ear,respectively, of the user.

This approach may be generalized to the synthesis of more complex soundfields according to

${{S_{i}\left( {k,l} \right)} = {\sum\limits_{j}{{H_{j,i}(k)}{S_{j}\left( {k,l} \right)}}}},{i = 1},2,$where S_(j)(k, l) is the STFT of the component of the sound of interestoriginating from location j, H_(j,i)(k) is the (personalized orHATS-based) HRTF from location j to the microphone close to the ith ear,and S_(i)(k, l) is the STFT of the sound to be presented to the ith ear.The location index j, could span some or all HRTFs in the combined HRTFset (i.e., both personal and HATS-based HRTFs). The advantage ofincluding personal HRTFs over using all-HATS-based HRTFs is that thespatial sound perception becomes more realistic to the individual user.

FIG. 3 shows a hearing aid system according to an embodiment of thepresent disclosure. The hearing aid system (HAS) comprises a hearing aid(HD) adapted for being worn by a user (U) at or in an ear of the user,or for being fully or partially implanted in the head at an ear of theuser. The hearing aid system (HAS) further comprises a portable orwearable auxiliary device (AD), e.g. a communication device, such as asmartphone, or a similar device, and/or an application program (APP)configured to be executed on the auxiliary device (cf. FIG. 4A, 4B). Thehearing aid system (HAS) is adapted to establish a communication link(WL-RF) between the hearing aid (HD) and the auxiliary device (AD) toprovide that data can be exchanged between them or forwarded from one tothe other.

The hearing aid (HD) comprises an input unit (IU) comprising at leastone microphone (here two, M₁, M₂) for picking up sound from theenvironment of the hearing aid and to provide corresponding at least oneelectric input signal (S₁(ω), S₂(ω)) representative of the sound (whereco may represent frequency). The input unit (IU) may comprise analogueto digital converters to provide the electric input signal(s) indigitized form as digital samples, and analysis filter banks forproviding the electric input signal(s) as frequency sub-band signals, asappropriate for the application in question. The hearing aid (HD)further comprises a signal processor (SPU) configured to performprocessing in the hearing aid. The signal processor (SPU) may comprise ahearing aid processor part (HAP) that is configured to process the atleast one electric input signal or a signal or signals derived therefromand to provide a processed signal (OUT). The hearing aid (HD) furthercomprises an output unit (OU), e.g. comprising a loudspeaker, avibrator, or a multi-electrode array, for presenting stimuli (e.g.acoustic vibrations or electric stimuli) perceivable as sound to theuser representative of the processed signal (OUT), see solid arrowdenoted ‘Stimuli’ in FIG. 3 from the output unit (OU) to the user (U).

The signal path from the input unit to the output unit (via hearing aidprocessor part (HAP)) may be denoted the ‘forward path’ in the hearingaid.

The auxiliary device (AD) comprises at least one microphone (AD-M) forpicking up sound from the environment of the auxiliary device (AD) andto provide corresponding at least one auxiliary electric input signal(ADM-IN) representative of the sound. The auxiliary device (AD) furthercomprises a user control interface (UI), e.g. a keyboard of a touchsensitive screen, allowing a user (U) to initiate a specific calibrationmode of operation of the hearing aid system (HAS), see solid arrowdenoted ‘V-Control’ and symbolic hand denoted ‘T-control’ in FIG. 3 fromthe user (U) to the user interface (UI) of the auxiliary device (AD). Asindicated in the embodiment of FIG. 3, the user control interface (UI)may comprise a touch sensitive display e.g. operated by one or morefingers (a stick, or the like) of the user (cf. ‘T-Control’ provided bythe finger(s) of a hand of the user), and/or it may comprise a voicecontrol interface reacting to spoken commands (cf. bold arrow denoted‘V-Control’ from the user's mouth to the user interface (UI)).

The embodiment of a hearing aid system in FIG. 3 comprises a memory(MEM) (here located in the hearing aid) for storing a time segment ofeach of the at least one electric input signals (here signals (S₁(ω),S₂(ω))). The memory (MEM) is also configured to store a time segment ofthe at least one auxiliary electric input signal (ADM-IN) represented bysignal (ADin) received from the auxiliary device (AD) via wirelesscommunication link (WL-RF), e.g. based on Bluetooth or similartechnology. The communication link is implemented by respective antennaand transceiver circuitry (TX/RX, ANT) of the two devices.

The signal processor (SPU) of the hearing aid (HD) is configured tocompare corresponding time segments of the at least one electric inputsignal (S₁(ω), S₂(ω)), and the at least one auxiliary electric inputsignal (ADin), or corresponding transforms thereof, and to provide anestimate of a transfer function (HRTF, OVTF) from the auxiliary device(AD) (e.g. from the at least one microphone (ADM) or from a loudspeaker(AD-SPK) of the auxiliary device, see below) to the at least onemicrophone (M₁, M₂) of the hearing aid (HD). In the embodiment of FIG.3, this task is performed by controller (TF-PRO) that receives a modecontrol signal (MCtr) from the user interface (UI) of the auxiliarydevice via the communication link (WL-RF). In the calibration mode ofoperation of the hearing aid system, the controller (TF-PRO) isconfigured to compare corresponding time segments (Tseg) of electricinput signals and based thereon to determine relevant acoustic transferfunctions of the system, as described in detail above. As outlined andexemplified above these acoustic transfer functions may e.g. be ownvoice transfer functions (OVTF) from the user's mouth to each of themicrophones (M₁, M₂). The own voice transfer functions (OVTF) may e.g.be used for customizing (personalizing) an own voice beamformer (OV-BF)of a beamformer filter that may be used to provide an estimate a user'svoice (OwnV) based on the two electric input signals S₁(ω), S₂(ω)) fromthe respective microphones (M₁, M₂) of the hearing aid (HD), e.g. foruse in a (subsequent) telephone mode of operation (cf. e.g. FIG. 5),where the estimate of the user's own voice is transmitted to a telephone(here e.g. the auxiliary device) for further transmission to afar-end-receiver via a communications network (e.g. a PSTN, or anInternet-based service, e.g. Skype, or WhatsApp). In the embodiment ofFIG. 3, the estimate of the user's voice (OwnV) is further processed(e.g. subject to a noise reduction algorithm, e.g. a post-filter) inprocessor part (OVP) providing an improved estimate of the user's ownvoice (OVest). In the embodiment of FIG. 3, the improved estimate of theuser's voice (OVest) is transmitted to the far-end-receiver. Theestimate of the user's voice may also be used in a keyword detector,e.g. in the hearing aid to support a voice control interface, or e.g. tocontrol functionality of the auxiliary device, or for being verified bya processor in the auxiliary device, or for being transmitted to aserver (e.g. in the Cloud) for further processing.

When the at least one microphone (ADM) of the auxiliary device (AD) ispositioned in proximity of, e.g. in front of, the user's mouth (as e.g.described in connection with FIG. 1A), a transfer function may representan own voice transfer function (OVTF), when the user—during thecalibration mode of operation of the hearing aid system—raises his orher voice, e.g. speaks (see time segment [t₁; t_(2]) in FIG. 1B). Thehearing aid system may be configured to instruct the user—e.g. via theuser interface (UI) of the auxiliary device (AD) (cf. e.g. FIG. 4B)—tospeak a certain phrase or sentence (e.g. with a particular vocal effort,e.g. loud or soft, etc., and/or of a specific duration, e.g. independence of a current noise level around the user).

The mode control signal (MCtr) from the user interface (UI) may e.g. beused to control the hearing aid signal processor (HAP) of the forwardpath of the hearing aid (HD) between the input unit (IU) and the outputunit (OU), cf. control signal HActr. In the embodiment of FIG. 3, theforward path further comprises a beamformer filter comprising a(far-field) beamformer (FF-BF) allowing a beamformed signal (YBF)focusing on a (far-field) communication partner (e.g. 1 m or more awayfrom the user's head (hearing aid(s))). The (far-field) beamformer isconnected to the hearing aid signal processor (HAP), which e.g. appliesone or more processing algorithms to the beamformed signal (YBF) (or asignal derived therefrom) and provides the processed signal (OUT), whichis fed to the output unit (OU) for presentation to the user. The one ormore processing algorithms may e.g. comprise one or more of noisereduction, compressive amplification (to compensate for the user'shearing impairment), feedback control, etc.

A customization (personalization) of the filter weights of the(far-field) beamformer (FF-BF) to the particular user may be performed(as described and exemplified in detail above) by the present embodimentof a hearing aid system using a loudspeaker of the auxiliary device toplay a test sound (calibration sound) in a specific calibration modewhose aim it is to determine head related transfer functions (HRTF, cf.e.g. FIG. 2), instead of own voice transfer functions (OVTF, cf. e.g.FIG. 1A).

The auxiliary device (AD) may thus (in an embodiment) preferablycomprise a loudspeaker (AD-SPK) and the auxiliary device may beconfigured to—in a specific calibration mode of operation—play a testsound signal (cf. ‘test sound, s_(f)(n) in FIG. 2) to the environment ofthe auxiliary device via said loudspeaker in dependence of an input fromthe user control interface (UI), cf. e.g. FIG. 4C.

In the calibration mode, the auxiliary device is positioned at apreferred location relative to the user (hearing aid microphone(s)) fromwhich an (acoustic) transfer function is to be estimated, e.g. held in ahand, or located at a table or other support. The preferred location(e.g. distance to, angle to, etc.) relative to the user may be known inadvance (e.g. carrying auxiliary device on a stick (e.g. a‘selfie-stick’) of known length), or be estimated during calibration,e.g. using one or more sensors, e.g. of the auxiliary device and/or thehearing aid, e.g. a camera, and/or a radar sensor. The hearing aidsystem (HAS) may be configured to make data representative of theestimated location of the loudspeaker (AD-SPK) relative to the hearingaid (HD) microphones (M₁, M₂) available (e.g. transmitted) to thehearing aid (e.g. via the communication link (WL-RF), and e.g. to formpart of the mode control signal (MCtr) fed to the controller (TF-PRO).

The auxiliary device (AD) comprises a controller (CNT) configured to—insaid specific (far-field) calibration mode of operation—provide a testor calibration signal (CalS), which is fed to and played by theloudspeaker (AD-SPK) thereby providing the test sound signal (cf. alsoFIG. 2). The controller (CNT) may comprise a test signal generator forproviding the test signal (CalS). The test signal may be(electromagnetically) transmitted directly to the hearing aid viacommunication link (WL-RF), cf. signal CalS' fed to the transceiver(TX/RX) of the auxiliary device. In the hearing aid, the wirelesslyreceived test signal (based on CalS′) may be represented by signal ADin,and—in the calibration mode of operation—stored in the memory (MEM)together with corresponding time segments of the electric input signals(S₁(ω), S₂(ω)) representing the (electric) versions of the test soundsignals received at the hearing aid microphones (M₁, M₂). Thereby thecontroller (TF-PRO) of the hearing aid may determine (frequencydependent) HRTFs for sound propagating from the current location of theloudspeaker (AD-SPK) relative to microphones (M₁, M₂) of the hearing aid(as described above).

The auxiliary device (AD) is configured to allow the control inputs(UCtr) from the user control interface (UI) to control the transmissionof microphone signals (ADM-IN) and/or test/calibration signals (CalS′)and/or other control signals (UCtr), e.g. mode control signals forinitiating and/or terminating a calibration mode, and/or other modes ofoperation of the hearing aid (e.g. a telephone mode) from the auxiliarydevice to the hearing aid(s).

In the embodiments of FIG. 1-3 communication between the hearing aid(HD) and the auxiliary device (AD) may be in the base band (audiofrequency range, e.g. between 0 and 20 kHz). Preferably however,communication between the hearing device and the auxiliary device isbased on some sort of modulation at frequencies above 100 kHz.Preferably, frequencies used to establish a communication link betweenthe hearing device and the auxiliary device is below 70 GHz, e.g.located in a range from 50 MHz to 70 GHz, e.g. above 300 MHz, e.g. in anISM range above 300 MHz, e.g. in the 900 MHz range or in the 2.4 GHzrange or in the 5.8 GHz range or in the 60 GHz range (ISM=Industrial,Scientific and Medical, such standardized ranges being e.g. defined bythe International Telecommunication Union, ITU). In an embodiment, thewireless link is based on a standardized or proprietary technology. Inan embodiment, the wireless link is based on Bluetooth technology (e.g.Bluetooth Low-Energy technology) or a related technology.

FIGS. 4A, 4B and 4C together illustrate an exemplary applicationscenario of an embodiment of a hearing system according to the presentdisclosure.

FIG. 4A illustrates a user, a binaural hearing aid system or hearingsystem and an auxiliary device during a calibration procedure (e.g. of aprocessing algorithm, e.g. a noise reduction system). FIG. 4A shows anembodiment of a head-worn binaural hearing aid system or a hearingsystem comprising left and right hearing devices (HA, HD_(r)) (e.g.hearing aids or ear pieces of a headset) in communication with aportable (handheld) auxiliary device (AD) functioning as a userinterface (UI) for the binaural hearing aid system or hearing system. Inan embodiment, the binaural hearing aid system or hearing systemcomprises the auxiliary device AD (and the user interface UI). Exemplaryscreens of the user interface UI of the auxiliary device AD are shown inFIGS. 4B and 4C. The user interface comprises a display (e.g. a touchsensitive display) displaying guidance to the user to select andinitiate (or terminate) a calibration mode of operation of the hearingaid system or hearing system. The user interface is implemented as anapplication program (APP) on the auxiliary device (e.g. a smartphone).The APP is denoted ‘Calibration APP. Activate calibration mode’. Via thedisplay of the user interface UI, the user U is instructed to selecteither Far-field calibration (HRTF), or Own voice calibration (OVTF).The calibration type is selected by pressing the ‘button’ in question,which when selected is indicated in grey shading, and instructions tothe user for carrying out the procedure for the particular calibrationare shown in the lower part of the screen. The individual screens forthe two calibration types are shown in FIGS. 4B and 4C, respectively.

FIG. 4B illustrates the auxiliary device AD running an APP forinitiating the calibration procedure for personalizing own voicetransfer functions.

The instructions for calibrating own voice transfer functions (OVTF) are

-   -   Locate device horizontally (microphone close to mouth).    -   During calibration keep your head still and don't move device.    -   Speak normally for ˜10 s.

These instructions should prompt the user to

-   -   Place the device with its microphone input close to the user's        mouth (e.g. ≤0.1 m from) while trying to minimize reflections of        the user's voice by the device (which may provide        reverberation-like disturbances and thus degrade the quality of        the OVTF-estimation).    -   Preferably, keep the device (and the body) as still as possible        during the length of the calibration, which is estimated at 10        seconds.    -   Speak normal sentences during the calibration period (e.g. with        a normal vocal effort). A further instruction may be to ask the        user to read a specific text that is known to ‘excite’ a        relevant frequency range of the user's voice.    -   Press Start/Stop ‘button’ to initiate calibration procedure.

FIG. 4C illustrates the auxiliary device running an APP for initiatingthe calibration procedure for personalizing head related transferfunctions.

The instructions for calibrating head related transfer functions (HRTF)are

-   -   Locate (e.g. hold) device at intended location with screen        towards you (loudspeaker at ear-level).    -   Activate selfie mode.    -   During calibration (while test sound is being played) keep your        head still and don't move device.

These instructions should prompt the user to

-   -   Place the auxiliary device in a location (direction and        distance) relative to the user where the target sound source is        expected to be located, e.g. in front of the user, e.g. ≥1 m        away from the user, e.g. by holding the auxiliary device in a        hand or on mounted on a stick (e.g. a ‘selfie-stick’).    -   Activate a camera mode of operation where the screen shows you a        ‘mirror-image of yourself’. This might help in positioning the        device in the right height (and may facilitate the use of        automatic positioning sensing using the camera image).        Preferably, the device should be at level with the eyes (and        ears) of the user.    -   Preferably, to keep the device (and the body) as still as        possible during the length of the calibration, which can be        verified by the user by the perception of the test sound (the        calibration procedure is e.g. estimated at 10 seconds). The        camera of the auxiliary device may record the user while the        sound is played (allowing an estimate of possible movements        during calibration).    -   Press Start/Stop ‘button’ to initiate calibration procedure.

The Start/Stop ‘button’ may further be used to terminate the calibrationprocedure, e.g. if something is not right (sudden movements, noise,other activities, etc.).

An acceptance step, requesting the user to accept the calibrationmeasurement may be included (to give the user a chance to discard theresults, if for some reason they are not as intended, e.g. due to noiseor other unintended events during the measurements).

Preferably, the initiation time of the calibration procedure (pressingof START) (and possibly the start time (and/or end time) of thecalibration signal), the chosen location (e.g. angle and distance to theuser), and possibly characteristics of the calibration signal (magnitudevs. frequency, spectrum, or the calibration signal itself (or a partthereof), etc.), are communicated to the left and right hearing devicesfor use in determining customized head related transfer functions (HRTF)or own voice transfer functions (OVTF). The customized (personalized)transfer functions may e.g. be used to choose an appropriatecorresponding (e.g. predetermined) set of filter weights, or forcalculating such weights, e.g. for an appropriate beamformer (cf. e.g.FF-BF and OV-BF in FIG. 3).

An example of an application of personalized transfer functionsaccording to the present disclosure is illustrated in FIG. 5. FIG. 5shows an embodiment of a headset or a hearing aid comprising own voiceestimation and the option of transmitting the own voice estimate toanother device, and to receive sound from another device forpresentation to the user via a loudspeaker, e.g. mixed with sound fromthe environment of the user. The hearing aid or headset (HD) (heretermed ‘hearing device’) comprises two microphones (M1, M2) configuredto provide electric input signals (IN1, IN2) representing sound in theenvironment of a user wearing the hearing device. The hearing devicefurther comprises two beamformers (FF-BF) and (OV-BF), each providing aspatially filtered signal (ENV and OV respectively) based on theelectric input signals (IN1, IN2) and personalized beamformer weightsaccording to the present disclosure. The (Far-field) beamformer (FF-BF)may e.g. implement a target maintaining, noise cancelling, beamformercomprising beamformer weights determined in dependence of personalizedacoustic transfer functions (HRTF) estimated according to the presentdisclosure. The own voice beamformer (OV-BF) is configured to pick upthe user's voice (originating from the user's mouth and throat) andcomprises beamformer weights determined in dependence of personalizedacoustic transfer functions (OV-TF) estimated according to the presentdisclosure. The hearing device may e.g. comprise an own voice detectorconfigured to detect whether or not (or with what probability) theuser's own voice is present (e.g. in the at least one electric inputsignal) at a given point in time, and to provide an own voice presencecontrol signal indicative thereof. The own voice beamformer (OV-BF) maye.g. be activated in dependence of an own voice presence control signal,and/or a telephone mode control signal, and/or a far-end talker presencecontrol signal, and/or a user-initiated control signal (e.g. via a userinterface (UI)). In a specific telephone mode of operation (e.g. anormal mode of operation of a headset), the user's own voice is pickedup by the microphones M1, M2 and spatially filtered by the own voicebeamformer (OV-BF) providing signal OV, which—optionally via own voiceprocessor (OVP) providing enhanced own voice signal (OVOUT)—is fed totransmitter Tx and transmitted (by cable or wireless link to a anotherdevice or system (e.g. a telephone, cf. dashed arrow denoted ‘To phone’and telephone symbol)). In the specific telephone mode of operation(e.g. a normal mode of operation of a headset), signal PHIN may bereceived by (wired or wireless) receiver Rx from another device orsystem (e.g. a telephone, as indicated by telephone symbol and dashedarrow denoted ‘From Phone’). When a far-end talker is active, signalPHIN contains speech from the far-end talker, e.g. transmitted via atelephone line (e.g. fully or partially wirelessly, but typically atleast partially cable-borne). The ‘far-end’ telephone signal PHIN may beselected or mixed with the environment signal ENV from the far-fieldbeamformer (FF-BF) in a combination unit (here selector/mixer SEL-MIX),and the selected or mixed signal PHENV is fed to output transducer SPK(e.g. a loudspeaker or a vibrator of a bone conduction hearing device)for presentation to the user as sound. Optionally, as shown in FIG. 5,the selected or mixed signal PHENV may be fed to processor PRO forapplying one or more processing algorithms to the selected or mixedsignal PHENV to provide processed signal OUT, which is then fed to theoutput transducer SPK. The embodiment of a hearing device (HD) of FIG. 5may represent a headset, in which case the received signal PHIN may beselected for presentation to the user without mixing with an environmentsignal. The embodiment of FIG. 5 may represent a hearing aid, in whichcase the received signal PHIN may be mixed with an environment signalbefore presentation to the user (to allow a user to maintain a sensationof the surrounding environment; the same may of course be relevant for aheadset application, depending on the use-case). Further, in a hearingaid, the processor (PRO) may be configured to compensate (signal PHENV)for a hearing impairment of the user of the hearing device (hearingaid).

It is intended that the structural features of the devices describedabove, either in the detailed description and/or in the claims, may becombined with steps of the method, when appropriately substituted by acorresponding process.

As used, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well (i.e. to have the meaning “at least one”),unless expressly stated otherwise. It will be further understood thatthe terms “includes,” “comprises,” “including,” and/or “comprising,”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof. It will also be understood that when an element is referred toas being “connected” or “coupled” to another element, it can be directlyconnected or coupled to the other element but an intervening element mayalso be present, unless expressly stated otherwise. Furthermore,“connected” or “coupled” as used herein may include wirelessly connectedor coupled. As used herein, the term “and/or” includes any and allcombinations of one or more of the associated listed items. The steps ofany disclosed method is not limited to the exact order stated herein,unless expressly stated otherwise.

It should be appreciated that reference throughout this specification to“one embodiment” or “an embodiment” or “an aspect” or features includedas “may” means that a particular feature, structure or characteristicdescribed in connection with the embodiment is included in at least oneembodiment of the disclosure. Furthermore, the particular features,structures or characteristics may be combined as suitable in one or moreembodiments of the disclosure. The previous description is provided toenable any person skilled in the art to practice the various aspectsdescribed herein. Various modifications to these aspects will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other aspects.

The claims are not intended to be limited to the aspects shown hereinbut are to be accorded the full scope consistent with the language ofthe claims, wherein reference to an element in the singular is notintended to mean “one and only one” unless specifically so stated, butrather “one or more.” Unless specifically stated otherwise, the term“some” refers to one or more.

Accordingly, the scope should be judged in terms of the claims thatfollow.

REFERENCES

-   [Farina, 2000]: Farina, Angelo. “Simultaneous measurement of impulse    response and distortion with a swept-sine technique.” Audio    Engineering Society Convention 108. Audio Engineering Society, 2000-   [Jensen et al., 2015]: J. Jensen and M. S. Pedersen, “Analysis of    Beamformer Directed Single-Channel Noise Reduction System for    Hearing Aid Applications”, Proc. Int. Conf. Acoust., Speech, Signal    Processing, pp. 5728-5732, April 2015.-   [Brandstein et al., 2001]: M. Brandstein and D. Ward (Eds.),    “Microphone Arrays—Signal Processing Techniques and Applications,”    Springer, 2001.-   [Haykin, 2001]: S. Haykin, “Adaptive Filter Theory”, Prentice Hall,    2001.-   [Heymann, et al., 2017] J. Heymann, L. Drude, R. Haeb-Umbach, “A    Generic Neural Acoustic Beamforming Architecture for Robust    Multi-Channel Speech Processing,” Computer, Speech and Language,    Vol. 46, pp. 374-385, November 2017.-   [Garde, 2019]. J. Garde, “Own-Voice Retrieval for Hearing Assistive    Devices: A Combined DNN-Beamforming Approach,” Master's Thesis,    Aalborg University, 2019.-   EP2928215A1 (Oticon) Jul. 10, 2015

The invention claimed is:
 1. A hearing aid system comprising a hearingaid adapted for being worn by a user at a or in an ear of the user, orfor being fully or partially implanted in the head at an ear of theuser, and a portable auxiliary device, wherein the hearing aid system isadapted to establish a communication link between the hearing aid andthe auxiliary device to provide that data can be exchanged between themor forwarded from one to the other, the hearing aid comprising an inputunit comprising at least one microphone for picking up sound from theenvironment of the hearing aid and to provide corresponding at least oneelectric input signal representative of said sound, a signal processorconfigured to process said at least one electric input signal or asignal or signals derived therefrom and to provide a processed signal,an output unit for presenting stimuli perceivable as sound to the userrepresentative of said processed signal, the auxiliary device comprisingat least one microphone for picking up sound from the environment of theauxiliary device and to provide corresponding at least one auxiliaryelectric input signal representative of said sound, a user controlinterface allowing a user to initiate a specific own voice calibrationmode of operation of the hearing aid system, wherein the signalprocessor is configured to receive corresponding time segments of saidat least one electric input signal, and said at least one auxiliaryelectric input signal, or corresponding transforms, or selectedfrequency regions thereof, and to provide an estimate of a personalizedtransfer function from said at least one microphone of said auxiliarydevice to said at least one microphone of said hearing aid, and whereinthe hearing aid comprises a beamformer filter comprising one or morebeamformers, the beamformer filter being configured to provide an ownvoice beamformer comprising personalized filter weights determined independence of said estimate of a personalized transfer function.
 2. Ahearing aid system according to claim 1 comprising a memory wherein timesegments of the at least one electric input signal, and/or the at leastone auxiliary electric input signal, or corresponding transforms, orselected frequency regions thereof, can be stored.
 3. A hearing aidsystem according to claim 1 wherein the auxiliary device is configuredto generate a calibration control signal upon initiation of the specificcalibration from the user interface.
 4. A hearing aid system accordingto claim 3 wherein the auxiliary device is configured to transmit acurrent time segment of the at least one auxiliary electric inputsignal, or corresponding transforms, or selected frequency regionsthereof, to the hearing aid in dependence of the calibration controlsignal.
 5. A hearing aid system according to claim 1 wherein aid atleast one microphone of the hearing aid comprises at least twomicrophones, wherein one of said at least one microphones of the hearingaid is defined as a reference microphone.
 6. A hearing aid systemaccording to claim 1 wherein the auxiliary device comprises aloudspeaker and wherein the auxiliary device is configured to—in aseparate far-field calibration mode of operation—play a test soundsignal to the environment of the auxiliary device and to transmit anelectric version of the test sound to the hearing aid in dependence ofan input from the user control interface.
 7. A hearing aid systemaccording to claim 6 wherein the hearing aid is configured to—in saidspecific far-field calibration mode of operation—store a time segment ofsaid at least one electric signal when said test sound signal is pickedup by said at least one microphone of the hearing aid.
 8. A hearing aidsystem according to claim 6 wherein the signal processor of the hearingaid—in said specific far-field calibration mode of operation—isconfigured to receive said electric version of the test sound and saidtime segment of said at least one electric signal and based thereon todetermine an acoustic transfer function from the loudspeaker of theauxiliary device to the at least one microphone of the hearing aid.
 9. Ahearing aid system according to claim 1 comprising a distance sensor forestimating a distance between the auxiliary device and the hearing aid.10. A hearing aid system according to claim 1 comprising a carrierconfigured to carry the auxiliary device.
 11. A hearing aid systemaccording to claim 1 wherein said one or more beamformers furthercomprises a beamformer comprising personalized filter weights, whereinthe beamformer is configured to suppress sound signals from a far-fieldspeaker.
 12. A hearing aid system according to claim 1 comprising twohearing aids adapted to implement a binaural hearing aid system, whereinthe two hearing aids comprise appropriate antenna and transceivercircuitry allowing them to exchange data between them.
 13. Use of ahearing aid system as claimed in claim 1 for determining personalizedparameters of a processing algorithm of the hearing aid.
 14. Anon-transitory application, termed an APP, comprising executableinstructions configured to be executed on an auxiliary device toimplement a user interface for a hearing aid system as claimed in claim1 wherein the user interface is configured to allow a user to controlfunctionality of the hearing aid system, including an initiation of aspecific calibration mode of operation of the hearing aid system.
 15. Anon-transitory application according to claim 14 wherein the APP isconfigured to run on a cellular phone or on another portable deviceallowing communication with said hearing aid or said hearing aid system.16. A method of operating a hearing aid system, the hearing aid systemcomprising a hearing aid adapted for being worn by a user at a or in anear of the user, or for being fully or partially implanted in the headat an ear of the user, and a portable auxiliary device, wherein thehearing aid system is adapted to establish a communication link betweenthe hearing aid and the auxiliary device to provide that data can beexchanged between them or forwarded from one to the other, the methodcomprising in the hearing aid providing by at least one microphone atleast one electric input signal representative of sound from theenvironment of the hearing aid, processing said at least one electricinput signal, or a signal or signals derived therefrom, and providing aprocessed signal, presenting stimuli perceivable as sound to the userrepresentative of said processed signal, in the auxiliary deviceproviding by at least one microphone at least one auxiliary electricinput signal representative of said sound from the environment of thehearing aid, providing a user control interface allowing the user toinitiate a specific own voice calibration mode of operation of thehearing aid system, and using corresponding time segments of said atleast one electric input signal, and said at least one auxiliaryelectric input signal, or corresponding transforms or selected frequencyregions thereof, to provide an estimate of a personalized transferfunction from said at least one microphone of said auxiliary device tosaid at least one microphone of said hearing aid, and providing an ownvoice beamformer comprising personalized filter weights determined independence of said estimate of a personalized transfer function.
 17. Ahearing aid system comprising at least one hearing aid adapted for beingworn by a user, and a portable auxiliary device, wherein the hearing aidsystem is adapted to establish a communication link between the at leastone hearing aid and the auxiliary device to provide that data can beexchanged between them, and wherein the at least one hearing aidcomprises at least one microphone for picking up sound from theenvironment of the hearing aid and configured to provide correspondingat least one electric input signal representative of the sound, andwherein the auxiliary device comprises at least one microphone forpicking up sound from the environment of the auxiliary device and toprovide corresponding at least one auxiliary electric input signalrepresentative of the sound, and wherein the hearing aid system furthercomprises a signal processor configured—in a specific own voicecalibration mode of operation of the hearing aid system— to comparecorresponding current time segments of the at least one electric inputsignal, and the at least one auxiliary electric input signal, orcorresponding transforms, or selected frequency ranges, thereof, whereinsaid corresponding current time segments are recorded while the userspeaks, and to provide an estimate of a personalized own voice transferfunction from the at least one microphone of the auxiliary device to theat least one microphone of the at least one hearing aid, when worn bythe user, and a beamformer filter configured to provide an own voicebeamformer comprising personalized filter weights determined independence of said estimate of a personalized own voice transferfunction.
 18. A hearing aid system according to claim 17 wherein thesignal processor is located in the at least one hearing aid, or in theportable auxiliary device, or in an external processing device.
 19. Ahearing aid system according to claim 18 wherein the at least onehearing aid comprises a hearing aid signal processor configured toprocess the electric input signal or a signal derived therefrom and toprovide a processed signal.
 20. A hearing aid system according to claim19 wherein the at least one hearing aid comprises an output transducerfor presenting stimuli perceivable as sound to the user.
 21. A hearingaid system according to claim 17 comprising a user control interfaceallowing a user to initiate the specific own voice calibration mode ofoperation of the hearing aid system.
 22. A hearing aid system accordingto claim 17 comprising one or more detectors configured to decidewhether or not, or with what probability, the user is currently wearingthe at least one hearing aid and to provide a mode control signalindicative thereof.
 23. A hearing aid configured to be used in a hearingaid system, wherein the hearing aid is configured to be worn by a userat a or in an ear of the user, or for being fully or partially implantedin the head at an ear of the user, the hearing aid comprising antennaand transceiver circuitry allowing the hearing aid to establish acommunication link to an auxiliary device to provide that data can beexchanged between them or forwarded from one to the other, at least onemicrophone for picking up sound from the environment of the hearing aidand for provide corresponding at least one electric input signalrepresentative of said sound, a signal processor configured to processsaid at least one electric input signal or a signal or signals derivedtherefrom and to provide a processed signal, and an output transducerfor presenting stimuli perceivable as sound to the user representativeof the processed signal, wherein the hearing aid is configured toreceive an auxiliary electric input signal provided by a microphone ofsaid auxiliary device via said communication link, and wherein thesignal processor—in a specific own voice calibration mode of operationof the hearing aid system—is configured to receive corresponding timesegments of the at least one electric input signal, and the at least oneauxiliary electric input signal, or corresponding transforms, orselected frequency ranges thereof, and to provide an estimate of apersonalized own voice transfer function from said microphone of saidauxiliary device to said at least one microphone of the hearing aid, andwherein the hearing aid further comprises a beamformer filter configuredto provide an own voice or an own-voice cancelling beamformer comprisingpersonalized filter weights determined in dependence of said estimate ofa personalized own voice transfer function.
 24. A hearing aid configuredto be used in a hearing aid system, wherein the hearing aid isconfigured to be worn by a user at a or in an ear of the user, or forbeing fully or partially implanted in the head at an ear of the user,the hearing aid comprising antenna and transceiver circuitry allowingthe hearing aid to establish a communication link to an auxiliary deviceto provide that data can be exchanged between them or forwarded from oneto the other, at least one microphone for picking up sound from theenvironment of the hearing aid and for provide corresponding at leastone electric input signal representative of said sound, a signalprocessor configured to process said at least one electric input signalor a signal or signals derived therefrom and to provide a processedsignal, and an output transducer for presenting stimuli perceivable assound to the user representative of the processed signal, wherein thehearing aid is configured to receive an auxiliary electric input signalprovided by a microphone of said auxiliary device via said communicationlink, and wherein the signal processor—in a specific own voicecalibration mode of operation of the hearing aid system—is configured toreceive corresponding time segments of the at least one electric inputsignal, and the at least one auxiliary electric input signal, orcorresponding transforms, or selected frequency ranges thereof, and toprovide an estimate of a personalized own voice transfer function fromsaid microphone of said auxiliary device to said at least one microphoneof the hearing aid, and wherein the hearing aid further comprises one ormore own-voice related algorithms optimized to a voice from a particularphysical person or to an artificial or recorded voice from a standardmodel, wherein the at least one microphone signal is equalized duringsignal segments where the own-voice signal dominates in dependence ofown voice transfer functions for said particular person or said modeland said user, respectively.